Intelligent multiple mode cooling unit for datacenter racks

ABSTRACT

A cooling system for a datacenter is disclosed. A multiple mode cooling subsystem of the cooling system has a form factor for one or more racks of the datacenter, has two or more different cooling systems, and is adjustable to different cooling requirements of the datacenter within different cooling capacities offered by the two or more different cooling systems.

FIELD

At least one embodiment pertains to cooling systems for a datacenter. Inat least one embodiment, a multiple mode cooling subsystem has a formfactor for one or more racks of the datacenter, has two or moredifferent cooling systems, and is adjustable to different coolingrequirements of the datacenter within different cooling capacitiesoffered by the two or more different cooling systems.

BACKGROUND

Datacenter cooling systems typically use fans to circulate air throughserver components. Certain supercomputers or other high capacitycomputers may use water or other cooling systems than air coolingsystems to draw heat away from the server components or racks of thedatacenter to an area external to the datacenter. The cooling systemsmay include a chiller within the datacenter area. The area external tothe datacenter may be a cooling tower or other external heat exchangerthat receives heated coolant from the datacenter and disperses the heatby forced air or other means to the environment (or an external coolingmedium) before the cooled coolant is recirculated back into thedatacenter. In an example, the chiller and the cooling tower togetherform a chilling facility with pumps responsive to temperature measuredby external devices applied to the datacenter. Air cooling systems alonemay not draw sufficient heat to support effective or efficient coolingin datacenters and liquid cooling systems may not be economical for thedemands of the datacenter.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIG. 1 is a block diagram of an example datacenter having a coolingsystem subject to improvements described in at least one embodiment;

FIG. 2 is a block diagram illustrating datacenter-level features of acooling system incorporating a multiple mode cooling subsystem,according to at least one embodiment;

FIG. 3A is a block diagram illustrating rack-level and server-levelfeatures of a cooling system incorporating an air-based coolingsubsystem of the multiple mode cooling subsystem, according to at leastone embodiment;

FIG. 3B is a block diagram illustrating rack-level and server-levelfeatures of a cooling system incorporating a liquid-based coolingsubsystem of the multiple mode cooling subsystem, according to at leastone embodiment;

FIG. 4A is a block diagram illustrating rack-level and server-levelfeatures of a cooling system incorporating a phase-based coolingsubsystem of the multiple mode cooling subsystem, according to at leastone embodiment;

FIG. 4B is a block diagram illustrating rack-level and server-levelfeatures of a cooling system incorporating a dielectric-based coolingsubsystem of the multiple mode cooling subsystem, according to at leastone embodiment;

FIG. 5 is a process flow of steps available for a method of using ormaking the cooling system of FIGS. 2-4B and 6A-17D, according to atleast one embodiment;

FIG. 6A illustrates an example datacenter, in which at least oneembodiment from FIGS. 2-5 may be used;

FIGS. 6B, 6C illustrate inference and/or training logic, such as used inFIG. 6A and in at least one embodiment of the present disclosure, forenabling and/or supporting a multiple mode cooling subsystem, accordingto various embodiments;

FIG. 7A is a block diagram illustrating an exemplary computer system,which may be a system with interconnected devices and components, asystem-on-a-chip (SOC) or some combination thereof formed with aprocessor that may include execution units to execute an instruction tosupport and/or to enable a multiple mode cooling subsystem describedherein, according to at least one embodiment;

FIG. 7B is a block diagram illustrating an electronic device forutilizing a processor to support and/or to enable a multiple modecooling subsystem described herein, according to at least oneembodiment;

FIG. 7C is a block diagram illustrating an electronic device forutilizing a processor to support and/or to enable a multiple modecooling subsystem described herein, according to at least oneembodiment;

FIG. 8 illustrates a further example computer system, according to atleast one embodiment, to implement various processes and methods for amultiple mode cooling subsystem described throughout this disclosure;

FIG. 9A illustrates an exemplary architecture in which GPUs arecommunicatively coupled to multi-core processors over high-speed linksfor enabling and/or supporting a multiple mode cooling subsystem,according to at least one embodiment of the disclosure herein;

FIG. 9B illustrates additional details for an interconnection between amulti-core processor and a graphics acceleration module in accordancewith one exemplary embodiment;

FIG. 9C illustrates another exemplary embodiment in which acceleratorintegration circuit is integrated within a processor for enabling and/orsupporting a multiple mode cooling subsystem, according to at least oneembodiment of the disclosure herein;

FIG. 9D illustrates an exemplary accelerator integration slice 990 forenabling and/or supporting a multiple mode cooling subsystem, accordingto at least one embodiment of the disclosure herein;

FIG. 9E illustrates additional details for one exemplary embodiment of ashared model. to enable and/or support a multiple mode coolingsubsystem, according to at least one embodiment of the disclosureherein;

FIG. 9F illustrates additional details for one exemplary embodiment of aunified memory, addressable via a common virtual memory address spaceused to access physical processor memories and GPU memories to enableand/or support a multiple mode cooling subsystem, according to at leastone embodiment of the disclosure herein;

FIG. 10A illustrates exemplary integrated circuits and associatedgraphics processors, according to embodiments described herein for amultiple mode cooling subsystem;

FIGS. 10B-10C illustrate exemplary integrated circuits and associatedgraphics processors, according to at least one embodiment, to supportand/or to enable a multiple mode cooling subsystem;

FIGS. 10D-10E illustrate additional exemplary graphics processor logic,according to at least one embodiment, to support and/or to enable amultiple mode cooling subsystem;

FIG. 11A is a block diagram illustrating a computing system to supportand/or to enable a multiple mode cooling subsystem according to at leastone embodiment;

FIG. 11B illustrates a parallel processor to support and/or to enable amultiple mode cooling subsystem according to at least one embodiment;

FIG. 11C is a block diagram of a partition unit according to at leastone embodiment;

FIG. 11D shows a graphics multiprocessor used for a multiple modecooling subsystem according to at least one embodiment;

FIG. 11E shows a graphics multiprocessor according to at least oneembodiment;

FIG. 12A illustrates a multi-GPU computing system, according to at leastone embodiment;

FIG. 12B is a block diagram of a graphics processor, according to atleast one embodiment;

FIG. 13 is a block diagram illustrating micro-architecture for aprocessor that may include logic circuits to perform instructions,according to at least one embodiment;

FIG. 14 illustrates a deep learning application processor, according toat least one embodiment;

FIG. 15 is a block diagram of a neuromorphic processor, according to atleast one embodiment;

FIG. 16A is a block diagram of a processing system, according to atleast one embodiment;

FIG. 16B is a block diagram of a processor having one or more processorcores, an integrated memory controller, and an integrated graphicsprocessor, according to at least one embodiment;

FIG. 16C is a block diagram of hardware logic of a graphics processorcore, according to at least one embodiment;

FIGS. 16D-16E illustrate thread execution logic including an array ofprocessing elements of a graphics processor core according to at leastone embodiment;

FIG. 17A illustrates a parallel processing unit, according to at leastone embodiment;

FIG. 17B illustrates a general processing cluster, according to at leastone embodiment;

FIG. 17C illustrates a memory partition unit of a parallel processingunit, in accordance with at least one embodiment; and

FIG. 17D illustrates a streaming multi-processor, according to at leastone embodiment.

DETAILED DESCRIPTION

Air cooling of high density servers is inefficient and ineffective inview of the high heat requirements caused by present day computingcomponents. However, the requirements are subject to change or tend torange from a minimum to a maximum of different cooling requirements. Thedifferent cooling requirements also reflect different heat features ofthe datacenter. In at least one embodiment, heat generated from thecomponents, servers, and racks are cumulatively referred to as a heatfeature or a cooling requirement as the cooling requirement must addressthe heat feature entirely. In at least one embodiment, the heat featureor the cooling requirement for a cooling system is the heat generated orthe cooling requirement of the components, servers, or racks associatedwith the cooling system and may be a portion of components, servers, andracks in the datacenter. For purposes of economies, providing aircooling along or providing liquid cooling alone may not address presentday datacenter cooling requirements. Further, providing liquid coolingwhen heat generated from a server is at a minimum of the differentcooling requirements taxes components of a liquid cooling system andalso wastes energy requirements to enable the liquid cooling system.Still further, the ability to address the different cooling requirementsis challenging within a mobile datacenter because of lack of space tosupport cooling systems able to provide a variety of different coolingmedia.

As such, the present disclosure seeks prospects in a cooling system fora datacenter having a multiple mode cooling subsystem in a form factorfor one or more racks of the datacenter. The one or more racks fitwithin a container or trailer that also includes the server racks of amobile datacenter. The multiple mode cooling subsystem has two or moredifferent cooling systems, and is adjustable to different coolingrequirements of the datacenter within different cooling capacitiesoffered by the two or more different cooling systems. The cooling systemis for cooling computing components such as a graphics processing unit(GPU), a central processing unit (CPU), or switching components. Thesecomputing components are used in servers assembled in server trays onracks in a datacenter. As the computing components are miniaturized bytechnology advances, the server trays and the racks accommodate more andmore computing components, thereby requiring dissipation of more heatgenerated per component than in prior systems. However, it is also thecase that computation and involvement of the computing components toperform functions cause the dissipation of more heat than when thecomputing components are idle. One issue addressed in the presentdisclosure is an inability to address the different cooling requirementsof the datacenter.

A datacenter liquid cooling system is supported by a chiller plant orsystem that may expensive because it is designed with over-provisioningrequirements. The over-provisioning requirements may be due to lack ofproper control for removal of heat from the server, such as from one ormore components (such as GPU, CPU, etc.), from the collective componentsin the server boxes, or the collective components within the racks.Efficient use of high-thermal-capacity liquid cooling systems may beover shadowed by the many intermediate features in the cooling system,include liquid chillers, pumps, coolant distribution units (CDUs), andheat exchangers. The multiple mode cooling subsystem of a datacentercooling system of the present disclosure is able to provide differentcooling capacities to address the different cooling requirements usinghigh temperature facility water that may be used as coolant in theprimary and the secondary cooling loops for liquid-based cooling of highheat components and systems; a refrigerant for a phase-based coolingsubsystem; a dielectric-based cooling subsystem for immersion cooling;and the air-based cooling subsystem, depending on the different coolingrequirements. As such, the multiple mode cooling subsystem enablesswitching of different cooling system which may be used for peaks havingand for removal of regular to extreme heat due to either computationalloads or environmental conditions.

In at least one embodiment, an evaporator coil associated with acondensing unit is provided external to the datacenter for a phase-basedcooling subsystem. A refrigerant changes phase from liquid to vapor inan evaporator to reach cooler temperatures within circulating pipes (orafter absorbing heat from the environment), which enables thephase-based cooling. Once the vapor is expended in dissipating coolertemperatures (or absorbing the heat), it is compressed and condensed ina condensing unit to a liquid phase and sent back to the evaporator. Inat least one embodiment, the condensing unit is as close to a secondarycooling loop as possible. In at least one embodiment, the secondarycooling loop for each of the liquid-based and the phase-based coolingsubsystems are neighborly, within their own pipes and manifolds. In atleast one embodiment, the condensing unit is located on a roof area ofthe datacenter or a container (of a mobile datacenter) and is able todirectly access room or row manifolds to provide refrigerant in arefrigerant cooling loop of a phase-based cooling subsystem. In at leaston embodiment, the evaporator coil enables a direct expansionrefrigerant for the refrigeration cooling loop and is co-existent with asecondary cooling loop carrying coolant, also referred to as the warmtemperature liquid cooling system.

A server box or server tray may include the above-referenced computingcomponents, but multiple server boxes, at a rack level or multiple ofracks benefit from the multiple mode cooling subsystem of the presentdisclosure. The multiple mode cooling subsystem is able to address thedifferent cooling requirements from the cooling subsystems fitted withina rack or multiple racks as a form factor of the cooling system. Thiseliminates additional costs to operate a datacenter chiller with moreefficiency than in cooler or nominal environmental temperatures, wherean air-based cooling system is sufficient. In at least one embodiment,miniature pumps or other flow control devices form the one or more flowcontrollers that are sensitive to the thermal requirements of at leastone or more sensors in the server or the racks for the multiple modecooling subsystem.

FIG. 1 is a block diagram of an example datacenter 100 having a coolingsystem subject to improvements described in at least one embodiment. Thedatacenter 100 may be one or more rooms 102 having racks 110 andauxiliary equipment to house one or more servers on one or more servertrays. The datacenter 100 is supported by a cooling tower 104 locatedexternal to the datacenter 100. The cooling tower 104 dissipates heatfrom within the datacenter 100 by acting on a primary cooling loop 106.Further, a cooling distribution unit (CDU) 112 is used between theprimary cooling loop 106 and a second or secondary cooling loop 108 toenable extraction of the heat from the second or secondary cooling loop108 to the primary cooling loop 106. The secondary cooling loop 108 isable to access various plumbing all the way into the server tray asrequired, in an aspect. The loops 106, 108 are illustrated as linedrawings, but a person of ordinary skill would recognize that one ormore plumbing features may be used. In an instance, flexible polyvinylchloride (PVC) pipes may be used along with associated plumbing to movethe fluid along in each of the loops 106, 108. One or more coolantpumps, in at least one embodiment, may be used to maintain pressuredifferences within the loops 106, 108 to enable the movement of thecoolant according to temperature sensors in various locations, includingin the room, in one or more racks 110, and/or in server boxes or servertrays within the racks 110.

In at least one embodiment, the coolant in the primary cooling loop 106and in the secondary cooling loop 108 may be at least water and anadditive, for instance, glycol or propylene glycol. In operation, eachof the primary and the secondary cooling loops has their own coolant. Inan aspect, the coolant in the secondary cooling loops may be proprietaryto requirements of the components in the server tray or racks 110. TheCDU 112 is capable of sophisticated control of the coolants,independently or concurrently, in the loops 106, 108. For instance, theCDU may be adapted to control the flow rate so that the coolant(s) isappropriately distributed to extract heat generated within the racks110. Further, more flexible tubing 114 is provided from the secondarycooling loop 108 to enter each server tray and to provide coolant to theelectrical and/or computing components. In the present disclosure, theelectrical and/or computing components are used interchangeably to referto the heat-generating components that benefit from the presentdatacenter cooling system. The tubing 118 that form part of thesecondary cooling loop 108 may be referred to as room manifolds.Separately, the tubing 116 extending from tubing 118 may also be part ofthe secondary cooling loop 108 but may be referred to as row manifolds.The tubing 114 enters the racks as part of the secondary cooling loop108, but may be referred to as rack cooling manifold. Further, the rowmanifolds 116 extend to all racks along a row in the datacenter 100. Theplumbing of the secondary cooling loop 108, including the manifolds 118,116, and 114 may be improved by at least one embodiment of the presentdisclosure. An optional chiller 120 may be provided in the primarycooling loop within datacenter 102 to support cooling before the coolingtower. To the extent additional loops exist in the primary control loop,a person of ordinary skill would recognize reading the presentdisclosure that the additional loops provide cooling external to therack and external to the secondary cooling loop; and may be takentogether with the primary cooling loop for this disclosure.

In at least one embodiment, in operation, heat generated within servertrays of the racks 110 may be transferred to a coolant exiting the racks110 via flexible tubing of the row manifold 114 of the second coolingloop 108. Pertinently, second coolant (in the secondary cooling loop108) from the CDU 112, for cooling the racks 110, moves towards theracks 110. The second coolant from the CDU 112 passes from on one sideof the room manifold having tubing 118, to one side of the rack 110 viarow manifold 116, and through one side of the server tray via tubing114. Spent second coolant (or exiting second coolant carrying the heatfrom the computing components) exits out of another side of the servertray (such as at the entry left side of the rack and exits right side ofthe rack for the server tray after looping through the server tray orthrough components on the server tray). The spent second coolant thatexits the server tray or the rack 110 comes out of different side (suchas at the exiting side) of tubing 114 and moves to a parallel, but alsoexiting side of the row manifold 116. From the row manifold 116, thespent second coolant moves in a parallel portion of the room manifold118 going in the opposite direction than the incoming second coolant(which may also be the renewed second coolant), and towards the CDU 112.

In at least one embodiment, the spent second coolant exchanges its heatwith a primary coolant in the primary cooling loop 106 via the CDU 112.The spent second coolant is renewed (such as being relatively cooledwhen compared to the temperature at the spent second coolant stage) andready to be cycled back to through the second cooling loop 108 to thecomputing components. Various flow and temperature control features inthe CDU 112 enable control of the heat exchanged from the spent secondcoolant or the flow of the second coolant in and out of the CDU 112. CDU112 is also able to control a flow of the primary coolant in primarycooling loop 106. As such, it is possible that some components withinthe servers and the racks do not receive the level of coolant requiredas the second or secondary loop provides coolant at its defaulttemperature properties based in part on the temperature sensors that maybe in the servers and the racks. Further, additional cooling loops maybe provided to exchange heat with the secondary cooling loop, but aresubject to similar restrictions by usage of a coolant as in the case ofthe primary or the secondary cooling loop.

FIG. 2 is a block diagram illustrating datacenter-level features of acooling system 200 incorporating a multiple mode cooling subsystem 206,according to at least one embodiment. The cooling system 200 is for adatacenter 202. The multiple mode cooling subsystem 206 includes two ormore of an air-based cooling system 228, a liquid-based coolingsubsystem 230, a phase-based cooling subsystem 232, or a dielectriccooling subsystem 234. The multiple mode cooling subsystem 206 is,therefore, rated for different cooling capacities as each of thesubsystems 228-234 may have a different rated cooling capacity. In atleast one embodiment, the air-based cooling system 228 is rated forbetween 10 kilowatt (KW or kW) to 30 KW of generated heat, theliquid-based cooling subsystem 230 is rated for between 30 KW to 60 KW,the phase-based cooling subsystem 234 is rated for between 60 KW to 100KW, and the dielectric cooling subsystem 232 is rated for between 100 KWto 450 KW of generated heat.

In at least one embodiment, the generated heat from is one or morecomputing component in servers of the racks of the datacenter. Therating is a feature of a cooling subsystem that can address (by coolingor by dissipating, in at least one embodiment) the generated heat whenit is at a range addressable by the cooling subsystem. Different coolingmedia are able to address the generated heat differently according tothe rating noted for the different cooling media. The generated heatfrom the one or more computing components, servers, or racks isdifferent than the ratings afforded to the cooling subsystems for theirindividual capability to address generated heat, and is referred to acooling requirement that must be addressed by the different coolingcapacities of the cooling subsystems.

In at least one embodiment, as the multiple mode cooling subsystem 206is rated for different cooling capacities because of at least thedifferent cooling media from the different cooling subsystems 228-234therein, the multiple mode cooling subsystem 206 is adjustable todifferent cooling requirements of the datacenter 202 within a minimumand a maximum of the different cooling capacities offered by thedifferent cooling subsystems 228-234. In at least one embodiment, thedifferent cooling media include air or gas, one or more different typesof coolant (including water and glycol-based coolants), and one or moredifferent types of refrigerant (including non-dielectric and dielectricrefrigerants). In at least one embodiment, the multiple mode coolingsubsystem 206 has a form factor for one or more racks 204 of thedatacenter. In at least one embodiment, the multiple mode coolingsubsystem 206 is in the form factor of one or more racks and isinstalled adjacent to one or more server racks 204 to address thedifferent cooling requirements of components and servers of the one ormore server racks 204.

In at least one embodiment, the one or more server racks 204 hasdifferent cooling requirements based in part on different heat generatedby components of server boxes or trays 220-226 within the one or moreserver racks 204. In at least one embodiment, multiple server boxes 220have nominal heat generated, which corresponds to about 10 KW to 30 KWof generated heat, and for which air-based cooling subsystem 228 isactivated to provide air-based cooling as illustrated by the directionalarrows from subsystem 228. The air-based cooling subsystem 228 sharescooling components with the liquid-based cooling subsystem 230 so thatthe air provided to the sever boxes 220 is a cooled air that may becooled by a heat exchanger within the air-based cooling subsystem 228.The heat exchanger within the air-based cooling subsystem 228 is part ofan air-based secondary cooling loop 242 that exchanges heat with acoolant-based primary cooling loop 238 associated with a CDU 236. In atleast one embodiment, a chilling facility formed of at least a chillerunit 208, a cooling tower 212, and a pump 210 supports the coolant-basedprimary cooling loop 238.

In at least one embodiment, the cooling system of the disclosureincludes a refrigerant-based heat transfer subsystem to enable one ormore of the phase-based cooling subsystem 232 and the dielectric coolingsubsystem 234; and includes a liquid-based heat transfer subsystem toenable one or more of the air-based cooling system 228 and theliquid-based cooling subsystem 230. In at least one embodiment, therefrigerant-based heat transfer subsystem includes at least pipes tosupport the refrigerant flow to the different cooling subsystems, andmay include one or more evaporative coils, compressors, condensers, andexpansion valves to support circulation of the refrigerant for thedifferent cooling subsystems. In at least one embodiment, acoolant-based heat transfer subsystem is provided to achieve similarcirculation of the coolant for the different cooling subsystems. In atleast one embodiment, a coolant-based coil subsystem and associatedwater spraying or fan-based cooling subsystem is included in thecoolant-based heat transfer subsystem. In at least one embodiment, adielectric refrigerant is used to enable the refrigerant-based coolingsubsystem 232 and the dielectric cooling subsystem 234. This enablesreduction in components required for the cooling system of the presentdisclosure, but supports a wide range of cooling capacities to addressthe different cooling requirements made of the system. Furthermore, inat least one embodiment, a coolant-based secondary cooling loop 244supports the liquid-based cooling subsystem 230.

In at least one embodiment, as a result of at least the aboveconfiguration, even though server trays 220 are air-cooled by air-basedcooling subsystem 228, server tray 222 may be liquid cooled (using acoolant, in at least one embodiment), via liquid-based cooling subsystem230; while server tray 224 may be immersion cooled (using a dielectricrefrigerant, in at least one embodiment), via the dielectric-basedcooling subsystem 234; and server tray 226 may be refrigerant cooled(using the same dielectric refrigerant or a separate refrigerant, in atleast one embodiment). In at least one embodiment, the refrigerant andthe dielectric cooling subsystems 232, 234 is directly coupled to atleast one condensing unit 214 to address the cooling requirement of therefrigerant via a refrigerant cooling loop 240.

In at least one embodiment, a coolant distributor, such as a coolantdistribution unit 236, provides coolant for the air-based coolingsubsystem 228 and the liquid-based cooling subsystem 230. In at leastone embodiment, for the liquid-based cooling subsystem 230, a primarycoolant cools a secondary coolant that circulates via the coolant-basedsecondary cooling loop 244 to cool components within the coolant-cooledserver tray 222. Although server tray 222 is referred to as acoolant-cooled server tray, it is appreciated that the coolant lines areavailable to all server trays 220-226 of rack 204 and to all racks ofthe datacenter 202 via at least row manifold 216, and respectiveinlet-side rack manifold(s) 248 and outlet side rack manifold(s) 246.

In at least one embodiment, the racks 204, 206 (including a rack orracks 206 forming the cooling subsystem) are on a raised platform orfloor 218, and the row manifold 216 may be a collection of manifoldswithin it to reach all the server trays or select server trays in theavailable racks of the datacenter. The row manifold 216 may be locatedabove the racks than as illustrated in FIG. 2. Similarly, in at leastone embodiment, for the refrigerant and immersion-based coolingsubsystems 232, 234, one or more of a dielectric refrigerant or aregular refrigerant circulates via a refrigerant-based secondary coolingloop 244 (that may extend from the primary cooling loop 240) to coolcomponents within the refrigerant and/or the immersion-cooled servertrays 224, 226. In at least one embodiment, although server trays 224,226 are referred to as refrigerant and/or the immersion-cooled servertrays 224, 226, it is appreciated that the refrigerant (or dielectricrefrigerant) lines are available to all server trays 220-226 of rack 204and to all racks of the datacenter 202 via at least row manifold 216,and respective inlet-side rack manifold(s) 248 and outlet side rackmanifold(s). The row manifold and the rack manifolds are illustrated assingle lines, but may be a collection of manifolds to carry coolants,refrigerants, and dielectric refrigerants distinctly, to reach all theserver trays or select server trays in the available racks of thedatacenter via one or more associated flow controllers. In this manner,in at least one embodiment, all the server trays may be able toseparately request (or indicate a requirement for) a desired level ofcooling from the multiple mode cooling subsystem 206.

In at least one embodiment, a first individual cooling subsystem, suchas the coolant-based cooling subsystem 230, of the multiple mode coolingsubsystem 206 has associated first looping components (such ascoolant-based secondary cooling loop in row manifold 216 and in rackmanifolds 246, 248) to guide a first cooling media (such as the coolant)to at least one first rack 204 of the datacenter 202 according to afirst heat feature or a cooling requirement of the at least one firstrack 204. In at least one embodiment, the multiple mode coolingsubsystem 206 has associated inlet and outlet lines 250 to circulate oneor more cooling media in and out of the appropriate manifolds of the rowmanifold 216. The first heat feature or the cooling requirement may be atemperature sensed by a sensor associated with the at least one firstrack 204. In at least one embodiment, concurrently with thecoolant-based cooling subsystem 230 cooling the at least one first rack204 (or a server therein), the refrigerant-based cooling subsystem 232of the multiple mode cooling subsystem 206 has associated second loopingcomponents (such as a refrigerant-based second cooling loop in rowmanifold 216 and in rack manifolds 246, 248) to guide a second coolingmedia (such as a refrigerant or dielectric refrigerant) to at least onesecond rack of the datacenter according to a second heat feature or asecond cooling requirement of the at least one second rack. This type ofdifferent cooling media addressing different requiring racks is alsopossible to address different requiring servers within the same rack asalready illustrated by rack 204 in FIG. 2.

In at least one embodiment, the first heat feature or the first coolingrequirement is within a first threshold of the minimum of the differentcooling capacities. In at least one embodiment, the minimum of thedifferent cooling capacities is provided by the air-based coolingsubsystem 228. In at least one embodiment, the second heat feature orthe second cooling requirement is within a second threshold of themaximum of the different cooling capacities. In at least one embodiment,the maximum of the different cooling capacities is provided by thedielectric-based cooling subsystem 234 that allows full immersion of thecomputing components in a dielectric media to cool the computingcomponents by direct convection cooling as opposed to air, coolant, andrefrigerant based media that work by radiative and/or conductivecooling. In at least one embodiment, cold plates associated with thecomputing components draw heat from the computing components andtransfers the heat to the coolant or refrigerant when these types ofcooling media are used. In at least one embodiment, the cold platestransfers the heat to air flowing through the cold plates (either havingassociated fins or directly via the cold plates) when the cooling mediais air.

In at least one embodiment, inlet and outlet lines 252, to rackmanifolds 246, 248 of the one or more racks 204, enable individualservers 220-226 associated with the rack manifold(s) 246, 248 to receivedifferent cooling media from individual cooling subsystems 230-234 ofthe multiple mode cooling subsystem 206. In at least one embodiment, oneor more servers 220-224 have or support two or more of an airdistribution unit, a coolant distribution unit, and a refrigerantdistribution unit to enable different cooling media to address thedifferent cooling requirements of the one or more servers. FIGS. 3A-4Baddresses server-level features of such servers, but briefly, acoolant-based heat exchanger (a coolant-based cold plate orcoolant-lines supported cold plate, in at least one embodiment) may beused as a coolant distribution unit for the server 222; arefrigerant-based heat exchanger (refrigerant-based cold plate orrefrigerant-lines supported cold plate, in at least one embodiment) maybe used as a refrigerant distribution unit for the server 224, and oneor more fans may be used as an air distribution unit for the servers220.

FIG. 3A is a block diagram illustrating rack-level 300A and server-level300B features of a cooling system incorporating an air-based coolingsubsystem of the multiple mode cooling subsystem, according to at leastone embodiment. In at least one embodiment, the air-based coolingsubsystem is able to address an entire rack 302 by blowing cold air 310through the entire rack 302 and cycling hot air 312 received from theopposite end of the rack 302. In at least one embodiment, one or morefans are provided in a conduit or an air manifold 318 to circulate thecold air 314, which may be the cold air 310 at the entry of the rack302, from the air-based cooling subsystem to the individual server 304.Hot air 316 flows out from an opposite end of the server 304 to join thehot air 312 exiting the rack 302. As such, server 304 is one or more ofservers 306 in rack 302, and has one or more components associated withcold plates 308A-D and requiring a nominal level of cooling. The coldplates 308A-D may have associated fins to enable more heat dissipationfor beneficial use of the air cooling media.

In at least one embodiment, the components associated with cold plates308A-D are computing components generating between 50 KW to 60 KW ofheat and sensors associated with the one or more servers, or the rackentirely, may indicate the temperate (representing a coolingrequirement) to the multiple mode cooling subsystem, which can thendetermine the specific one of the cooling subsystems 228-234 to activatefor the one or more servers or the rack entirely. In at least oneembodiment, the indication of the temperature may be sent to the coolingsubsystems 228-234 and the cooling subsystem designated to address aspecific temperature corresponding to KW heat generated (for thespecific temperature) activates itself to address the server or theracks requiring the cooling. This represents a distributed controlsystem, where control of activation or deactivation is shared byprocessors of the different subsystems 228-234. In at least oneembodiment, a centralized control system can be enabled by one or moreprocessors of a server 306 in the server rack 302. The one or moreprocessors is adapted to perform the controlling requirements for themultiple mode cooling system, including by using artificial intelligenceand/or machine learning to train a learning subsystem to recognizetemperature indications and to activate or deactivate appropriatecooling subsystems 228-234. The distributed control system may be alsoadapted to use artificial intelligence and/or machine learning to traina learning subsystem, albeit by a distributed neural network, forinstance.

FIG. 3B is a block diagram illustrating rack-level 330A and server-level330B features of a cooling system incorporating a liquid-based coolingsubsystem of the multiple mode cooling subsystem, according to at leastone embodiment. In at least one embodiment, the liquid-based coolingsubsystem is a coolant-based cooling subsystem. The rack-level features330A supporting or incorporating the liquid-based cooling subsystemincludes a rack 332 having inlet and outlet coolant-based manifolds358A, 358B, having associated therewith inlet and outlet couplers oradapters 354, 356, and having associated therewith individual flowcontrollers 362 (two are referenced in the figure). The inlet and outletcouplers or adapters 354, 356 receive, direct, send coolant 352 throughone or more requesting (or indicating) servers 336; 304. In at least oneembodiment, all the servers 336 in the rack 332 may be addressed bytheir individual requests (or indications) for the same coolingrequirement, or by a request (or indication) from the rack 332 on behalfof its servers 336, which can be addressed by the liquid-based coolingsubsystem. Heated coolant 360 exits the rack via adapter or coupler 356.The individual flow controllers 362 may be pumps that are controlled(activated, deactivated, or maintained in flow position) via inputs fromthe distributed or centralized control system, in at least oneembodiment. Further, in at least one embodiment, when the rack 332 makesits request (or provides an indication) for a cooling requirement, allthe flow controllers 362 may be simultaneously controlled to address thecooling requirement, as if the cooling requirement is representative ofcooling requirements of all the servers 336 at the same time. In atleast one embodiment, the request or the indication comes from a sensorassociated with the rack that senses temperature within the rack.

In at least one embodiment, the request or the indication is made from asensor associated with the rack or with the servers. There may be one ormore sensors in different locations of the rack and in differentlocations of the server. The sensors, in a distributed configuration,may send individual requests or indications of a temperature, in atleast one embodiment. In at least one embodiment, the sensors work in acentralized configuration and are adapted to send its requests orindications to one sensor or to a centralized control system thataggregates the requests or indications to determine a singular action.In at least one embodiment, the singular action may be to send one ormore cooling media from the multiple mode cooling subsystem to addressthe requirements or indications associated with the singular action. Inat least one embodiment, this enables the present cooling system to sendcooling media associated with a higher cooling capacity or capabilitythan the aggregate cooling requirement of some of the servers in theracks as the aggregated requested or indications may include at leastsome cooling requirements that are higher than the aggregate coolingrequirement.

In at least one embodiment, a mode statistical measure is used as anaggregate to indicate the number of servers having higher coolingrequirement, which may be addressed instead of a pure average of thecooling requirements of the servers. As such, it is appreciated that asubsystem offering higher cooling capacity (such as refrigerant-basedcooling subsystem) may be activated when at least 50% of the servers ofa rack have higher cooling requirements. In at least one embodiment, asubsystem offering lower cooling capacity (such as coolant-based coolingsubsystem) may be activated when at most 50% of the servers of the rackhave higher cooling requirements. However, when a number of the serversof the rack start requesting or indicating higher cooling requirementsafter the coolant-based cooling subsystem is activated for the rack,then a higher cooling capacity may be provided to address the shortfall.In at least one embodiment, the change of the specific cooling subsystemmay occur to higher or lower capacity cooling based in part on an amountof time elapsed since a first request or indication occurred and whenthe first request or indicate persists or shows an elevation of anassociated temperature.

In at least one embodiment, the servers 336 have individual server-levelmanifolds, such as manifold 338 in server 334 (also referred to asserver tray or server box). The manifold 338 of server 334 includes oneor more server-level flow controllers 340A, B for addressing individualcomponent(s) requirements for cooling. The server manifold 338 receivescoolant from rack manifold 358A (via adapter or coupler 348) and,depending on requirements of the computing components associated withcold plates 342A-D, is caused to distribute the coolant (or anassociated server-level coolant exchanging heat with the rack levelcoolant) to the components via the associated cold plates 342A-D. In atleast one embodiment, the distributed or centralized control systemassociated with the multiple mode cooling subsystem is able to controlthe rack level flow controllers 362 and the server level flowcontrollers 340A, B to facilitate movement of coolant from theliquid-based cooling subsystem. As such, in at least one embodiment, asensor on a first side of the server 334 having components associatedwith a first set of cold plates 342A, B may indicate or request highercooling requirements than a sensor on a second side of the server 334having components associated with a second set of cold plates 342C, D.Then the coolant, if a coolant is sent as cooling media, is enabled toflow, via flow controller 340A and inlet line 344A, through the firstset of cold plates 342A, B, out via outlet line 346A, and back to theserver manifold 338. The heated coolant may exit the server manifold viaadapter or coupler 350.

In at least one embodiment, the server manifold 338 as well as the rowmanifold(s) 358A, B include multiple sub-manifolds or lines andassociated couplers to support the other cooling media, including therefrigerant (or the dielectric refrigerant), but is illustrated and isdiscussed here for the coolant (liquid-based media) portion as relevantto this feature. In at least one embodiment, as the sensor on the secondside of the server 334 having components associated with the second setof cold plates 342C, D did not indicate or request higher coolingrequirements than the sensor on the first side of the server 334 havingcomponents associated with the first set of cold plates 342A, B, theflow controller 340B remains closed or unchanged (if previously openedfor a different cooling media). There will be no change to or no coolingmedia in inlet line 344B, the second set of cold plates 342C, D, and theoutlet line 346B to the server manifold 338.

FIG. 4A is a block diagram illustrating rack-level 400A and server-level400B features of a cooling system incorporating a phase-based coolingsubsystem of the multiple mode cooling subsystem, according to at leastone embodiment. In at least one embodiment, the phase-based coolingsubsystem is a refrigerant-based cooling subsystem. The rack-levelfeatures 400A supporting or incorporating the phase-based coolingsubsystem includes a rack 402 having inlet and outlet phase-basedmanifolds 428A, 428B, having associated therewith inlet and outletcouplers or adapters 424, 426, and having associated therewithindividual flow controllers 432 (two are referenced in the figure). Theinlet and outlet couplers or adapters 424, 426 receive, direct, sendrefrigerant 422 through one or more requesting (or indicating) servers406; 404. In at least one embodiment, all the servers 406 in the rack402 may be addressed by their individual requests (or indications) forthe same cooling requirement, or by a request (or indication) from therack 402 on behalf of its servers 406, which can be addressed by thephase-based cooling subsystem. Liquid phase refrigerant may enter therack manifold via adapter or coupler 424 and vapor phase refrigerant 430exits the rack via adapter or coupler 426. The individual flowcontrollers 432 may be pumps that are controlled (activated,deactivated, or maintained in flow position) via inputs from thedistributed or centralized control system as discussed in reference toFIGS. 2, 3A, and 3B, in at least one embodiment. Further, in at leastone embodiment, when the rack 402 makes its request (or provides itsindication) for a cooling requirement, all the flow controllers 432 maybe simultaneously controlled to address cooling requirement as if thecooling requirement is representative of cooling requirements of all theservers 406, at the same time. In at least one embodiment, the requestor the indication comes from a sensor associated with the rack thatsenses temperature within the rack.

In at least one embodiment, the request or the indication is made from asensor associated with the rack or with the servers. As in theembodiment of FIG. 3B, there may be one or more sensors in differentlocations of the rack and in different locations of the server. Thesensors, in a distributed configuration, may send individual requests orindications of a temperature, in at least one embodiment. In at leastone embodiment, the sensors work in a centralized configuration and areadapted to send its requests or indications to one sensor or to acentralized control system that aggregates the requests or indicationsto determine a singular action. In at least one embodiment, the singularaction may be to send one or more cooling media from the multiple modecooling subsystem to address the requirements or indications associatedwith the singular action. In at least one embodiment, this enables thepresent cooling system to send cooling media associated with a highercooling capacity or capability than the aggregate cooling requirement ofsome of the servers in the racks as the aggregated requested orindications may include at least some cooling requirements that arehigher than the aggregate cooling requirement.

In at least one embodiment, a mode statistical measure is used as anaggregate to indicate the number of servers having higher coolingrequirement, which may be addressed instead of a pure average of thecooling requirements of the servers. As such, it is appreciated that asubsystem offering higher cooling capacity or capability (such asimmersion-based cooling subsystem) may be activated when at least 50% ofthe servers of a rack have higher cooling requirements. However, theserver requesting higher cooling capacity or capability must alsosupport the cooling subsystem offering the higher cooling capacity orcapability. In at least one embodiment, a server supporting immersioncooling must have a sealed space in which to allow a dielectric coolingmedia, such as a dielectric refrigerant to contact components directly.In at least one embodiment, a subsystem offering lower cooling capacityor capability (such as the refrigerant or a coolant-based coolingsubsystem) may be activated when at most 50% of the servers of the rackhave higher cooling requirements. However, when a number of the serversof the rack start requesting or indicating higher cooling requirementsafter the refrigerant or a coolant-based cooling subsystem is activatedfor the rack, then a higher cooling capacity or capability may beprovided to address the shortfall. In at least one embodiment, thechange of the specific cooling subsystem may occur to higher or lowercapacity or capability cooling based in part on an amount of timeelapsed since a first request or indication occurred and when the firstrequest or indicate persists or shows an elevation of an associatedtemperature.

In at least one embodiment, the servers 406 have individual server-levelmanifolds, such as manifold 408 in server 404. The manifold 408 ofserver 404 includes one or more server-level flow controllers 410A, Bfor addressing individual component(s) requirements for cooling. Theserver manifold 408 receives liquid phase refrigerant from rack manifold428A (via adapter or coupler 418) and, depending on requirements of thecomputing components associated with cold plates 412A-D, is caused todistribute the refrigerant to the components via the associated coldplates 412A-D. In at least one embodiment, the distributed orcentralized control system associated with the multiple mode coolingsubsystem is able to control the rack level flow controllers 432 and theserver level flow controllers 410A, B to facilitate movement ofrefrigerant from the phase-based cooling subsystem. As such, in at leastone embodiment, a sensor on a first side of the server 404 havingcomponents associated with a first set of cold plates 412A, B mayindicate or request higher cooling requirements than a sensor on asecond side of the server 404 having components associated with a secondset of cold plates 4122C, D. Then the refrigerant, if a refrigerant issent as cooling media, is enabled to flow, via flow controller 410A andinlet line 414A, through the first set of cold plates 412A, B, out viaoutlet line 416A, and back to the server manifold 408. The vapor phaserefrigerant may exit the server manifold via adapter or coupler 420.

In at least one embodiment, the server manifold 404 as well as the rowmanifold(s) 428A, B include multiple sub-manifolds or lines andassociated couplers to support the other cooling media, including thecoolant (or the dielectric refrigerant), but is illustrated and isdiscussed here for the refrigerant (phase-based media) portion asrelevant to this feature. In at least one embodiment, as the sensor onthe second side of the server 404 having components associated with thesecond set of cold plates 412C, D did not indicate or request highercooling requirements than the sensor on the first side of the server 404having components associated with the first set of cold plates 412A, B,the flow controller 410B remains closed or unchanged (if previouslyopened for a different cooling media). There will be no change to or nocooling media in inlet line 414B, the second set of cold plates 412C, D,and the outlet line 416B to the server manifold 408.

FIG. 4B is a block diagram illustrating rack-level 450A and server-level450B features of a cooling system incorporating a dielectric-basedcooling subsystem of the multiple mode cooling subsystem, according toat least one embodiment. In at least one embodiment, thedielectric-based cooling subsystem is an immersion-based coolingsubsystem. The dielectric-based cooling subsystem enables immersioncooling of computing components in one or more servers 456 of the racks452. The rack-level features 450A supporting or incorporating thedielectric-based cooling subsystem includes a rack 452 having inlet andoutlet dielectric-based manifolds 478A, 478B, having associatedtherewith inlet and outlet couplers or adapters 474, 476, and havingassociated therewith individual flow controllers 482 (two are referencedin the figure). In at least one embodiment, when the dielectric coolingmedia is a dielectric refrigerant, then the cooling componentsassociated with the refrigerant-based cooling subsystem may be used tosupport the dielectric-based cooling subsystem. The inlet and outletcouplers or adapters 474, 476 receive, direct, send dielectric coolingmedia 472 through one or more requesting (or indicating) servers 456;454.

In at least one embodiment, all the servers 456 in the rack 452 may beaddressed by their individual requests (or indications) for the samecooling requirement, or by a request (or indication) from the rack 452on behalf of its servers 456, which can be addressed by thedielectric-based cooling subsystem. Dielectric cooling media 480 exitsthe rack via adapter or coupler 476. The individual flow controllers 483may be pumps that are controlled (activated, deactivated, or maintainedin flow position) via inputs from the distributed or centralized controlsystem, in at least one embodiment. Further, in at least one embodiment,when the rack 452 makes its request (or provides its indication, from anassociated sensor, in at least one embodiment) for a coolingrequirement, all the flow controllers 482 may be simultaneouslycontrolled to address the cooling requirement, as if the coolingrequirement is representative of cooling requirements of all the servers456 at the same time.

In at least one embodiment, the request or the indication is made from asensor associated with the rack or with the servers. There may be one ormore sensors in different locations of the rack and in differentlocations of the server. The sensors, in a distributed configuration,may send individual requests or indications of a temperature, in atleast one embodiment. In at least one embodiment, the sensors work in acentralized configuration and are adapted to send its requests orindications to one sensor or to a centralized control system thataggregates the requests or indications to determine a singular action.In at least one embodiment, the singular action may be to send one ormore cooling media from the multiple mode cooling subsystem to addressthe requirements or indications associated with the singular action. Inat least one embodiment, this enables the present cooling system to sendcooling media associated with a higher cooling capability than theaggregate cooling requirement of some of the servers in the racks as theaggregated requested or indications may include at least some coolingrequirements that are higher than the aggregate cooling requirement.

In at least one embodiment, a mode statistical measure is used as anaggregate to indicate the number of servers having higher coolingrequirement, which may be addressed instead of a pure average of thecooling requirements of the servers. As such, it is appreciated that asubsystem offering higher cooling capability (such as immersion orrefrigerant-based cooling subsystem) may be activated when at least 50%of the servers of a rack have higher cooling requirements. In at leastone embodiment, a subsystem offering lower cooling capability (such as arefrigerant or a coolant-based cooling subsystem) may be activated whenat most 50% of the servers of the rack have higher cooling requirements.However, when a number of the servers of the rack start requesting orindicating higher cooling requirements after the refrigerant orcoolant-based cooling subsystem is activated for the rack, then a highercooling capability (such as from the dielectric-based cooling subsystem)may be provided to address the shortfall, if the servers (and computingcomponents) at issue can support immersion cooling. In at least oneembodiment, the change of the specific cooling subsystem may occur tohigher or lower capability cooling based in part on an amount of timeelapsed since a first request or indication occurred and when the firstrequest or indicate persists or shows an elevation of an associatedtemperature.

In at least one embodiment, the servers 456 have individual server-levelmanifolds, such as manifold 458 in server 454 (also referred to asserver tray or server box). As the server requires specific adaptationto support immersion cooling, such as being hermetically sealed, in atleast one embodiment. The manifold 458 of server 454 is adapted to floodthe server box for immersion cooling. The server manifold 338 receivesthe dielectric-based cooling media from rack manifold 478A (via adapteror coupler 468) and, depending on requirements of the computingcomponents associated with cold plates 462A-D, is caused to flood thedielectric-based cooling media into the server 454, so that thecomponents are directly in contact with the dielectric-based coolingmedia. As such, in at least one embodiment, the cold plates 462A-D maynot be required. In at least one embodiment, the distributed orcentralized control system associated with the multiple mode coolingsubsystem is able to control the rack level flow controllers 482 tofacilitate movement of dielectric-based cooling media from thedielectric-based cooling subsystem. As such, in at least one embodiment,a sensor associated with the cold plates 462A-D may indicate or requesthigher cooling requirements than presently supported by, any of thecoolant, refrigerant, or air-based cooling media. Then thedielectric-based cooling media is enabled to flow, via the appropriateflow controller 482 and via inlet adapter or coupler 468, to the server454, and out of outlet adapter or coupler 470. Each of the inlet and theoutlet adapter or coupler 468; 470 includes a one-way valve, so that thedielectric-based cooling media flows into the server box through oneadapter or coupler 468 and out of the other adapter or coupler 470.

In at least one embodiment, the server manifold 458 as well as the rowmanifold(s) 478A, B include multiple sub-manifolds or lines andassociated couplers to support the other cooling media, including thecoolant and the refrigerant (if the dielectric cooling media is not thesame as the refrigerant), but is illustrated and is discussed here forthe dielectric-based cooling media portion as relevant to this feature.In at least one embodiment, dielectric-based cooling media may enter therack manifold via adapter or coupler 474 and the heated or useddielectric-based cooling media 480 exits the rack via adapter or coupler476.

FIG. 5 is a process flow of steps available for a method 500 of using ormaking the cooling system of FIGS. 2-4B and 6A-17D, according to atleast one embodiment. In at least one embodiment, the method for coolinga datacenter includes a step 502 for providing a multiple mode coolingsubsystem that is rated for different cooling capacities. In at leastone embodiment, the step 502 provides the multiple mode coolingsubsystem with two or more of an air-based cooling system, aliquid-based cooling subsystem, a dielectric cooling subsystem, or aphase-based cooling subsystem. The multiple mode cooling subsystem istherefore rated for the different cooling capacities based in part onthe ratings for each of the cooling subsystems. Furthermore, themultiple mode cooling subsystem is provided in a form factor for one ormore racks of the datacenter. In at least one embodiment, the multiplemode cooling subsystem is in the form factor of one or more racks.

In at least one embodiment, the method 500 includes step 504 thatenables adjustment of the multiple mode cooling subsystem so that thedifferent cooling subsystems within the multiple mode cooling subsystemis able to be activated and deactivated to operate independently of eachother or concurrently with each other. Step 506 determines, from sensorsdeployed at various locations of the servers or the racks, indicationsof different temperatures. In at least one embodiment, the indication isa request to a receiving component, such as a distributed or acentralized control system to react to the different temperatures or tocontinue monitoring the different temperatures.

In at least one embodiment, the distributed or a centralized controlsystem is enabled to, via step 508, perform a determination for whetherthe different temperatures reflect different cooling requirements withina minimum and a maximum of the different cooling capacities of themultiple mode cooling subsystem. For instance, the distributed or acentralized control system has stored therein adaptable values oftemperatures corresponding to different temperature ranges that may besupported by each of the air-based cooling system, the liquid-basedcooling subsystem, the dielectric cooling subsystem, or the phase-basedcooling subsystem. The adaptable values of temperatures reflectdifferent cooling capabilities of the different cooling subsystems. Theair-based cooling subsystem addresses the least cooling requirement ofthe datacenter, which also reflects a minimum cooling capability of themultiple mode cooling subsystem. In at least one embodiment, thedielectric-based cooling subsystem reflects a maximum cooling capabilityof the multiple mode cooling subsystem. Further, in at least oneembodiment, when servers are unlikely to support the dielectric-basedcooling subsystem, then a refrigerant (or other phase)-based coolingsubsystem, at its highest cooling capability end, reflects the maximumcooling capability of the multiple mode cooling subsystem.

Step 510, based in part on a positive determination from step 508, thatthe different temperatures from sensors in the servers or racks reflectdifferent cooling requirements within a minimum and a maximum of thedifferent cooling capacities, the multiple mode cooling subsystem may beactivated. In step 510, as part of the activation of the multiple modecooling subsystem, the distributed or a centralized control system isadapted to provide different cooling media from the different coolingsubsystems according to the cooling requirements of the differentservers or racks. In at least one embodiment, the distributed or acentralized control system has stored therein adaptable values of flowrates or volumes for the different cooling media to cause flowcontrollers associated with the different cooling subsystems to activateindependent or simultaneously to address the cooling requirements of thedifferent servers or racks. A negative determination from step 508enables on-going or continuous monitoring of the sensors in step 506.

In at least one embodiment, as part of the distributed control system,steps 506-510 occur via processors associated with each flow controllerof the multiple mode cooling subsystem. The steps 506-510, performed ina distributed manner, enables the flow controllers associated with thedifferent cooling subsystems of the multiple mode cooling subsystem toreceive input from temperature sensors, reflecting the step 508, wherethe sensors are associated with one or more components within servers ofthe one or more racks of the datacenter. The steps 506-510 performed viathe distributed control system enables the flow controllers tofacilitate movement of two different cooling media by controlling theflow controllers associated with the two different cooling media toaddress the different cooling requirements.

In at least one embodiment, step 502 includes providing the multiplemode cooling subsystem by enabling the phase-based cooling subsystemusing a refrigerant-based heat transfer subsystem and enabling theliquid-based cooling subsystem and the dielectric cooling subsystemusing a liquid-based heat transfer subsystem. In at least oneembodiment, step 510 includes providing the multiple mode coolingsubsystem by enabling the liquid-based cooling subsystem and thedielectric cooling subsystem using a dielectric coolant. In at least oneembodiment, steps 506-510 guide a first cooling media to at least onefirst rack of the datacenter according to a first heat feature or acooling requirement of the at least one first rack using a firstindividual cooling subsystem of the multiple mode cooling subsystemhaving associated first looping components. Concurrently, steps 506-510also guide a second cooling media to at least one second rack of thedatacenter according to a second heat feature or a cooling requirementof the at least one second rack using a second individual coolingsubsystem of the multiple mode cooling subsystem having associatedsecond looping components.

In at least one embodiment, steps 506-510 enable one or more first flowcontrollers to maintain a first cooling media of different cooling mediaassociated with the multiple mode cooling subsystem at a first flow ratebased in part on a first cooling requirement of the different coolingrequirements. Concurrently, these steps also enable one or more secondflow controllers to modify a second cooling media of the differentcooling media to flow at one or more flow rates based in part on asecond cooling requirement of the different cooling requirements that issensed from within one or more of servers of the datacenter.

In at least one embodiment, a flow controller may be provided on an exitside of the piping in any of the datacenter features. Furthermore, aflow controller may be provided on both the entry and the exit sides ofthe piping in any of the datacenter features. As such, when the flowcontroller is on the exit side, it performs a suction action rather thana pushing action. When two flow controllers are working in tandem, thereare both, suction and pushing actions. The flow rates achieved may behigher by the tandem flow controllers. The configuration or adaption ofthe flow controllers may be determined by requirements of thecomponents, for instance.

In at least one embodiment, the coolant-based secondary cooling loop ofthe embodiment in FIG. 3A facilitates a default or standard movement ofcoolant in a component-level cooling loop (such as enabled by piping344A, 346A in the server 334) or of a second coolant if the secondarycooling loop is directly available to a component. In at least oneembodiment, a learning subsystem may be implemented via the deeplearning application processor, such as processor 1400 in FIG. 14 andmay use the neurons 1502 and components thereof implemented usingcircuitry or logic, including one or more arithmetic logic units (ALUs)as described in FIG. 15. As such, the learning subsystem includes atleast one processor for evaluating temperatures within servers of theone or more racks with flow rates of different cooling media. Thelearning subsystem also provides outputs associated with at least twotemperatures for facilitating movement of two cooling media of thedifferent cooling media by controlling flow controllers associated withthe two cooling media to address the different cooling requirements.

Once trained, the learning subsystem will be able to provide one or moreoutputs having instructions associated with flow rates for the differentcooling media to the flow controllers. The outputs are provided after anevaluation of the prior associated flow rates of individual coolingmedia of the different cooling media and prior associated temperatures,as described in reference to FIGS. 2-5. In at least one embodiment, thetemperatures used in the learning subsystem may be an inferred valuefrom a voltage and/or current value output by a respective component,such as a sensor, but the voltage and/or the current value itself may beused to train the learning subsystem, along with the requisite flow raterequired of the different cooling media to cause one or more computingcomponents, servers, racks, or associated sensors to achieve thetemperatures. Alternatively, the temperatures reflect one or moretemperatures at which the requisite flow rates for the cooling media isrequired of the cooling media, over what is already provided by thecooling media to cool the one or more computing components, servers,racks, or associated sensors, and subsequent temperatures may be used inthe learning subsystem to reflect the temperatures at which therequisite flow rates for the requisite cooling media are no longerrequired.

Alternatively, in at least one embodiment, the difference in thetemperatures and the subsequent temperatures of the different coolingmedia from the different cooling subsystems, along with the requisiteflow rates is used to train the learning system to recognize when toactivate and deactivate associated flow controllers for the differentcooling media. Once trained, the learning subsystem, through a devicecontroller (also referred to herein as a centralized control system ordistributed control system) described elsewhere throughout thisdisclosure, is able to provide one or more outputs to control theassociated flow controllers of the different cooling subsystems inresponse to temperature received from temperature sensors associatedwith different components, different servers, and different racks.

Furthermore, the learning subsystem executes a machine learning modelthat processes temperatures collected from prior applications, perhapsin a test environment, to control the temperatures within the differentservers and racks, and associated with different components. Thecollected temperatures may include (a) achieved temperatures for certainflow rates of the different cooling media; (b) achieved difference intemperatures for certain flow rates of the different cooling media forspecific time periods; and (c) initial temperatures and correspondingflow rates of the different media applied to keep the different servers,racks, or components in optimal operation by their different coolingrequirements. In at least one embodiment, the collected information mayinclude a reference to a coolant or a refrigerant type, or acompatibility type (such as a server or a rack having immersion-coolingcompatibility), which reflects the type of cooling required oravailable.

In at least one embodiment, aspects of the processing for the deeplearning subsystem may use the collected information processed in linewith the features discussed with reference to FIGS. 14, 15. In anexample, the processing of the temperatures uses multiple neuron levelsof the machine learning model that are loaded with one or more of thecollected temperature features noted above and the corresponding flowrates of the different cooling media. The learning subsystem performs atraining that may be represented as an evaluation of temperature changesassociated with prior flow rates (or changes in flow rates) of eachcooling media as per adjustments made to the one or more flowcontrollers associated with the different cooling media. The neuronlevels may store values associated with the evaluation process and mayrepresent an association or correlation between the temperature changesand the flow rates for each of the different cooling media.

In at least one embodiment, the learning subsystem, once trained, isable to determine, in application, flow rates of the different mediarequired to achieve cooling to a temperature (or a change, such as areduction in temperature) associated with cooling requirements ofdifferent servers, different racks, or different components, forinstance. As the cooling requirements have to be within the coolingcapabilities of the appropriate cooling subsystem, the learningsubsystem is able to select which of the cooling subsystems to activateto address the temperature sensed from the different servers, thedifferent racks, or the different components. The collected temperaturesand prior associated flow rates for the different cooling media that isused to achieve the collected temperatures (or differences, forinstance) may be used by the learning subsystem to provide one or moreoutputs associated with required flow rates of different cooling mediato address the different cooling requirements reflected by a temperature(representing a reduction) than a present temperature for the differentservers, the different racks, or the different components.

In at least one embodiment, a result of the learning subsystem is, inresponse to a sensed temperature from the different servers, thedifferent racks, or the different components, one or more outputs to theflow controllers associated with the different cooling subsystems thatmodifies flow rates of the respective different cooling media. Themodification of the flow rates enables a determined flow of therespective different cooling media to reach the different servers, thedifferent racks, or the different components requiring the cooling. Themodified flow rate may be maintained till the temperature in therequiring area reaches a temperature associated with the flow rate ofthe cooling media that is known to the learning subsystem. In at leastone embodiment, the modified flow rate may be maintained till thetemperature in the area changes by a determined value. In at least oneembodiment, the modified flow rate may be maintained till thetemperature in the area reaches a rated temperature for the differentservers, different racks, the different components, or the differentcooling media.

In at least one embodiment, the device controller (also referred to as acentralized or distributed control system) includes at least oneprocessor having at least one logic unit to control flow controllersassociated with the different cooling subsystems. In at least oneembodiment, the device controller may be at least one processor withinthe datacenter, such as a processor 702 of FIG. 7A. The flow controllerfacilitates movement of a respective cooling media associated with arespective cooling subsystem, and facilitates cooling of areas in thedatacenter in response to temperatures sensed in the area. In at leastone embodiment, the at least one processor is a processor core of amulti-core processor, such as multi-core processors 905, 906 in FIG. 9A.In at least one embodiment, the at least one logic unit may be adaptedto receive a temperature values from temperature sensors associated withservers or the one or more racks, and is adapted to facilitate movementof a first individual cooling media and a second individual coolingmedia of the at least two cooling media.

In at least one embodiment, a processor, such as the processor cores ofmulti-core processors 905, 906 in FIG. 9A may include a learningsubsystem for evaluating temperatures from sensors in differentlocations in the datacenter, such as in different locations associatedwith the servers or the racks, or even a component within the server,with flow rates that are associated with at least two cooling media of amultiple mode cooling subsystem having two or more of an air-basedcooling subsystem, a liquid-based cooling subsystem, a dielectric-basedcooling subsystem, or a phase-based cooling subsystem. The learningsubsystem provides an output, such as an instruction that is associatedwith at least two temperatures for facilitating movement of at least twocooling media by controlling the flow controllers associated with the atleast two cooling media to address the different cooling requirements.

In at least one embodiment, the learning subsystem executes a machinelearning model to process the temperature using multiple neuron levelsof the machine learning model having the temperatures and having priorassociated flow rates for the different cooling media. The machinelearning model may be a implemented using the neurons structuredescribed in FIG. 15 and the deep learning processor as described inFIG. 14. The machine learning model provides the output associated withthe flow rate, from an evaluation of the prior associated flow rates, tothe one or more flow controllers. Furthermore, an instruction output ofthe processor, such as a pin of a connector bus or a ball of a ball gridarray, enables communication of the output with the one or more flowcontrollers to modify a first flow rate of a first cooling media of theat least two different cooling media, while maintaining a second flowrate of a second individual cooling media.

In at least one embodiment, the present disclosure is to at least oneprocessor for a cooling system or to a system having the at least oneprocessor. The at least one processor includes at least one logic unitto train a neural network having hidden layers of neurons for evaluatingtemperatures and flow rates that are associated with at least twocooling media of a multiple mode cooling subsystem comprising two ormore of an air-based cooling subsystem, a liquid-based coolingsubsystem, a dielectric-based cooling subsystem, or a phase-basedcooling subsystem. The multiple mode cooling subsystem is rated fordifferent cooling capacities and is adjustable to different coolingrequirements of the datacenter within a minimum and a maximum of thedifferent cooling capacities. The cooling system has a form factor forone or more racks of a datacenter.

As described elsewhere in this disclosure and with references made toFIGS. 2-5, 14, and 15, the training may be performed by layers ofneurons that are provided inputs of the temperatures of one or moreareas in the datacenter and associated flow rates of different coolingmedia from prior application, perhaps in a test environment. Thetemperatures may include a starting temperature (and associated flowrate of each cooling media applied for the starting temperature to bringthe temperature to a rated temperature of the one or more areas), afinal temperature (after each cooling media is applied, over eachcooling media already in the area, for a flow rate for a period oftime), and a difference in temperatures achieved for the flow rate andthe period of time of application of the different cooling media. One ormore of these features may be used to train a neural network todetermine when to apply a flow of the different cooling media and/orwhen to stop the flow of the different cooling media for cooling—such aswhen a temperature sensed for an area reaches the starting temperature,when the temperature sensed for the area reaches the final temperature,and/or when the temperature sensed for the area reflects the differencein temperature suited for the area (such as a reduction in temperatureto a rated temperature).

In at least one embodiment, the at least one processor includes at leastone logic unit configured or adapted for training a neural network andis a multi-core processor. In at least one embodiment, the at least onelogic unit may be within one processor core that can be used to forevaluate temperatures and flow rates that are associated with at leasttwo cooling media of a multiple mode cooling subsystem having two ormore of an air-based cooling subsystem, a liquid-based coolingsubsystem, a dielectric-based cooling subsystem, or a phase-basedcooling subsystem. The multiple mode cooling subsystem is rated fordifferent cooling capacities and is adjustable to different coolingrequirements of the datacenter within a minimum and a maximum of thedifferent cooling capacities. The cooling system has a form factor forone or more racks of a datacenter.

In at least one embodiment, the present disclosure enables all-in-oneair, liquid, refrigeration, and immersion cooling system that is in aform factor of an individual server rack or multiple racks and thatprovides required cooling with an air-based cooling subsystem having aheat exchanger associated with a coolant, with a warm or cold waterliquid-based cooling subsystem for chip cooling, with a refrigeration toheat exchanger phase-based cooling subsystem, and with an immersionblade cooling reflecting a dielectric-based cooling subsystem. Themultiple mode cooling subsystem in a rack is part of a cooling systemthat can provide cooling by passing air through cooled heat exchangers,but can also provide cooling water from a primary liquid supply, and canalso provide a vapor compression cooling system for refrigerationcooling with an associated evaporator heat exchanger for either serveror chip cooling. In addition, a coolant or refrigerant that is ofdielectric properties can be used either from the liquid-based coolingsubsystem or the refrigerant (or phase)-based cooling subsystem forremoving thermal load from an immersion blade cooling server in a rack.

In at least one embodiment, the present disclosure is therefore anall-in-one rack cooling system that can provide cooling from arelatively low density rack having about 10 KW to higher density coolingof about 30 KW using the air-base cooling subsystem; from about 30 KW to60 KW using a liquid-based cooling subsystem; from about 60 KW to 100 kWusing the phase-based cooling subsystem, and from about 100 KW to 450 KWusing the dielectric-based cooling subsystem for immersion cooling. Theall-in-one rack is an all in one structure and reflects a foot printthat can be readily deployed in almost all data centers.

Datacenter

FIG. 6A illustrates an example datacenter 600, in which at least oneembodiment from FIGS. 2-5 may be used. In at least one embodiment,datacenter 600 includes a datacenter infrastructure layer 610, aframework layer 620, a software layer 630, and an application layer 640.In at least one embodiment, such as described in respect to FIG. 2,features in the components of the cooling subsystems of the multiplemode cooling subsystem may be performed inside or in collaboration withthe example datacenter 600. In at least one embodiment, theinfrastructure layer 610, the framework layer 620, the software layer630, and the application layer 640 may be partly or fully provided viacomputing components on server trays located in racks 210 of thedatacenter 200. This enables cooling systems of the present disclosureto direct cooling to certain ones of the computing components in anefficient and effective manner. Further, aspects of the datacenter,including the datacenter infrastructure layer 610, the framework layer620, the software layer 630, and the application layer 640 may be usedto support the refrigerant-assisted cooling discussed with at leastreference to FIGS. 2-5 above. As such, the discussion in reference toFIGS. 6A-17D may be understood to apply to the hardware and softwarefeatures required to enable or support the multiple mode coolingsubsystem for the datacenter of FIGS. 2-5, for instance.

In at least one embodiment, as in FIG. 6A, datacenter infrastructurelayer 610 may include a resource orchestrator 612, grouped computingresources 614, and node computing resources (“node C.R.s”)616(1)-616(N), where “N” represents any whole, positive integer. In atleast one embodiment, node C.R.s 616(1)-616(N) may include, but are notlimited to, any number of central processing units (“CPUs”) or otherprocessors (including accelerators, field programmable gate arrays(FPGAs), graphics processors, etc.), memory devices (e.g., dynamicread-only memory), storage devices (e.g., solid state or disk drives),network input/output (“NW I/O”) devices, network switches, virtualmachines (“VMs”), power modules, and cooling modules, etc. In at leastone embodiment, one or more node C.R.s from among node C.R.s616(1)-616(N) may be a server having one or more of above-mentionedcomputing resources.

In at least one embodiment, grouped computing resources 614 may includeseparate groupings of node C.R.s housed within one or more racks (notshown), or many racks housed in datacenters at various geographicallocations (also not shown). Separate groupings of node C.R.s withingrouped computing resources 614 may include grouped compute, network,memory or storage resources that may be configured or allocated tosupport one or more workloads. In at least one embodiment, several nodeC.R.s including CPUs or processors may grouped within one or more racksto provide compute resources to support one or more workloads. In atleast one embodiment, one or more racks may also include any number ofpower modules, cooling modules, and network switches, in anycombination.

In at least one embodiment, resource orchestrator 612 may configure orotherwise control one or more node C.R.s 616(1)-616(N) and/or groupedcomputing resources 614. In at least one embodiment, resourceorchestrator 612 may include a software design infrastructure (“SDI”)management entity for datacenter 600. In at least one embodiment,resource orchestrator may include hardware, software or some combinationthereof.

In at least one embodiment, as shown in FIG. 6A, framework layer 620includes a job scheduler 622, a configuration manager 624, a resourcemanager 626 and a distributed file system 628. In at least oneembodiment, framework layer 620 may include a framework to supportsoftware 632 of software layer 630 and/or one or more application(s) 642of application layer 640. In at least one embodiment, software 632 orapplication(s) 642 may respectively include web-based service softwareor applications, such as those provided by Amazon Web Services, GoogleCloud and Microsoft Azure. In at least one embodiment, framework layer620 may be, but is not limited to, a type of free and open-sourcesoftware web application framework such as Apache Spark™ (hereinafter“Spark”) that may utilize distributed file system 628 for large-scaledata processing (e.g., “big data”). In at least one embodiment, jobscheduler 622 may include a Spark driver to facilitate scheduling ofworkloads supported by various layers of datacenter 600. In at least oneembodiment, configuration manager 624 may be capable of configuringdifferent layers such as software layer 630 and framework layer 620including Spark and distributed file system 628 for supportinglarge-scale data processing. In at least one embodiment, resourcemanager 626 may be capable of managing clustered or grouped computingresources mapped to or allocated for support of distributed file system628 and job scheduler 622. In at least one embodiment, clustered orgrouped computing resources may include grouped computing resource 614at datacenter infrastructure layer 610. In at least one embodiment,resource manager 626 may coordinate with resource orchestrator 612 tomanage these mapped or allocated computing resources.

In at least one embodiment, software 632 included in software layer 630may include software used by at least portions of node C.R.s616(1)-616(N), grouped computing resources 614, and/or distributed filesystem 628 of framework layer 620. One or more types of software mayinclude, but are not limited to, Internet web page search software,e-mail virus scan software, database software, and streaming videocontent software.

In at least one embodiment, application(s) 642 included in applicationlayer 640 may include one or more types of applications used by at leastportions of node C.R.s 616(1)-616(N), grouped computing resources 614,and/or distributed file system 628 of framework layer 620. One or moretypes of applications may include, but are not limited to, any number ofa genomics application, a cognitive compute, and a machine learningapplication, including training or inferencing software, machinelearning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.) orother machine learning applications used in conjunction with one or moreembodiments.

In at least one embodiment, any of configuration manager 624, resourcemanager 626, and resource orchestrator 612 may implement any number andtype of self-modifying actions based on any amount and type of dataacquired in any technically feasible fashion. In at least oneembodiment, self-modifying actions may relieve a datacenter operator ofdatacenter 600 from making possibly bad configuration decisions andpossibly avoiding underutilized and/or poor performing portions of adatacenter.

In at least one embodiment, datacenter 600 may include tools, services,software or other resources to train one or more machine learning modelsor predict or infer information using one or more machine learningmodels according to one or more embodiments described herein. In atleast one embodiment, in at least one embodiment, a machine learningmodel may be trained by calculating weight parameters according to aneural network architecture using software and computing resourcesdescribed above with respect to datacenter 600. In at least oneembodiment, trained machine learning models corresponding to one or moreneural networks may be used to infer or predict information usingresources described above with respect to datacenter 600 by using weightparameters calculated through one or more training techniques describedherein. As previously discussed, deep learning techniques may be used tosupport intelligent control of the flow controllers in therefrigerant-assisted cooling by monitoring area temperatures of thedatacenter. Deep learning may be advanced using any appropriate learningnetwork and the computing capabilities of the datacenter 600. As such, adeep neural network (DNN), a recurrent neural network (RNN) or aconvolutional neural network (CNN) may be supported eithersimultaneously or concurrently using the hardware in the datacenter.Once a network is trained and successfully evaluated to recognize datawithin a subset or a slice, for instance, the trained network canprovide similar representative data for using with the collected data.

In at least one embodiment, datacenter 600 may use CPUs,application-specific integrated circuits (ASICs), GPUs, FPGAs, or otherhardware to perform training and/or inferencing using above-describedresources. Moreover, one or more software and/or hardware resourcesdescribed above may be configured as a service to allow users to trainor performing inferencing of information, such as pressure, flow rates,temperature, and location information, or other artificial intelligenceservices.

Inference and Training Logic

Inference and/or training logic 615 may be used to perform inferencingand/or training operations associated with one or more embodiments. Inat least one embodiment, inference and/or training logic 615 may be usedin system FIG. 6A for inferencing or predicting operations based, atleast in part, on weight parameters calculated using neural networktraining operations, neural network functions and/or architectures, orneural network use cases described herein. In at least one embodiment,inference and/or training logic 615 may include, without limitation,hardware logic in which computational resources are dedicated orotherwise exclusively used in conjunction with weight values or otherinformation corresponding to one or more layers of neurons within aneural network. In at least one embodiment, inference and/or traininglogic 615 may be used in conjunction with an application-specificintegrated circuit (ASIC), such as Tensorflow® Processing Unit fromGoogle, an inference processing unit (IPU) from Graphcore™, or aNervana® (e.g., “Lake Crest”) processor from Intel Corp.

In at least one embodiment, inference and/or training logic 615 may beused in conjunction with central processing unit (CPU) hardware,graphics processing unit (GPU) hardware or other hardware, such as fieldprogrammable gate arrays (FPGAs). In at least one embodiment, inferenceand/or training logic 615 includes, without limitation, code and/or datastorage modules which may be used to store code (e.g., graph code),weight values and/or other information, including bias values, gradientinformation, momentum values, and/or other parameter or hyperparameterinformation. In at least one embodiment, each of the code and/or datastorage modules is associated with a dedicated computational resource.In at least one embodiment, the dedicated computational resourceincludes computational hardware that further include one or more ALUsthat perform mathematical functions, such as linear algebraic functions,only on information stored in code and/or data storage modules, andresults from which are stored in an activation storage module of theinference and/or training logic 615.

FIGS. 6B, 6C illustrates inference and/or training logic, such as usedin FIG. 6A and in at least one embodiment of the present disclosure,according to at least one embodiment. The inference and/or traininglogic 615 are used to perform inferencing and/or training operationsassociated with at least one embodiment. Details regarding inferenceand/or training logic 615 are provided below in conjunction with FIGS.6B and/or 6C. The inference and/or training logic 615 of FIGS. 6B and 6Care distinguished by the use of the arithmetic logic units (ALUs) 610versus the computational hardware 602, 606. In at least one embodiment,each of computational hardware 602 and computational hardware 606includes one or more ALUs that perform mathematical functions, such aslinear algebraic functions, only on information stored in code and/ordata storage 601 and code and/or data storage 605, respectively, resultof which is stored in activation storage 620. As such, FIGS. 6B and 6Cmay be alternatives and may be used interchangeably unless statedotherwise.

In at least one embodiment, inference and/or training logic 615 mayinclude, without limitation, code and/or data storage 601 to storeforward and/or output weight and/or input/output data, and/or otherparameters to configure neurons or layers of a neural network trainedand/or used for inferencing in at least one embodiment. In at least oneembodiment, training logic 615 may include, or be coupled to code and/ordata storage 601 to store graph code or other software to control timingand/or order, in which weight and/or other parameter information is tobe loaded to configure, logic, including integer and/or floating pointunits (collectively, arithmetic logic units (ALUs). In at least oneembodiment, code, such as graph code, loads weight or other parameterinformation into processor ALUs based on an architecture of a neuralnetwork to which the code corresponds. In at least one embodiment, codeand/or data storage 601 stores weight parameters and/or input/outputdata of each layer of a neural network trained or used in conjunctionwith at least one embodiment during forward propagation of input/outputdata and/or weight parameters during training and/or inferencing usingat least one embodiment. In at least one embodiment, any portion of codeand/or data storage 601 may be included with other on-chip or off-chipdata storage, including a processor's L1, L2, or L3 cache or systemmemory.

In at least one embodiment, any portion of code and/or data storage 601may be internal or external to one or more processors or other hardwarelogic devices or circuits. In at least one embodiment, code and/or codeand/or data storage 601 may be cache memory, dynamic randomlyaddressable memory (“DRAM”), static randomly addressable memory(“SRAM”), non-volatile memory (e.g., Flash memory), or other storage. Inat least one embodiment, choice of whether code and/or code and/or datastorage 601 is internal or external to a processor, for example, orincluded of DRAM, SRAM, Flash or some other storage type may depend onavailable storage on-chip versus off-chip, latency requirements oftraining and/or inferencing functions being performed, batch size ofdata used in inferencing and/or training of a neural network, or somecombination of these factors.

In at least one embodiment, inference and/or training logic 615 mayinclude, without limitation, a code and/or data storage 605 to storebackward and/or output weight and/or input/output data corresponding toneurons or layers of a neural network trained and/or used forinferencing in at least one embodiment. In at least one embodiment, codeand/or data storage 605 stores weight parameters and/or input/outputdata of each layer of a neural network trained or used in conjunctionwith at least one embodiment during backward propagation of input/outputdata and/or weight parameters during training and/or inferencing usingat least one embodiment. In at least one embodiment, training logic 615may include, or be coupled to code and/or data storage 605 to storegraph code or other software to control timing and/or order, in whichweight and/or other parameter information is to be loaded to configure,logic, including integer and/or floating point units (collectively,arithmetic logic units (ALUs).

In at least one embodiment, code, such as graph code, loads weight orother parameter information into processor ALUs based on an architectureof a neural network to which the code corresponds. In at least oneembodiment, any portion of code and/or data storage 605 may be includedwith other on-chip or off-chip data storage, including a processor's L1,L2, or L3 cache or system memory. In at least one embodiment, anyportion of code and/or data storage 605 may be internal or external toon one or more processors or other hardware logic devices or circuits.In at least one embodiment, code and/or data storage 605 may be cachememory, DRAM, SRAM, non-volatile memory (e.g., Flash memory), or otherstorage. In at least one embodiment, choice of whether code and/or datastorage 605 is internal or external to a processor, for example, orincluded of DRAM, SRAM, Flash or some other storage type may depend onavailable storage on-chip versus off-chip, latency requirements oftraining and/or inferencing functions being performed, batch size ofdata used in inferencing and/or training of a neural network, or somecombination of these factors.

In at least one embodiment, code and/or data storage 601 and code and/ordata storage 605 may be separate storage structures. In at least oneembodiment, code and/or data storage 601 and code and/or data storage605 may be same storage structure. In at least one embodiment, codeand/or data storage 601 and code and/or data storage 605 may bepartially same storage structure and partially separate storagestructures. In at least one embodiment, any portion of code and/or datastorage 601 and code and/or data storage 605 may be included with otheron-chip or off-chip data storage, including a processor's L1, L2, or L3cache or system memory.

In at least one embodiment, inference and/or training logic 615 mayinclude, without limitation, one or more arithmetic logic unit(s)(“ALU(s)”) 610, including integer and/or floating point units, toperform logical and/or mathematical operations based, at least in parton, or indicated by, training and/or inference code (e.g., graph code),a result of which may produce activations (e.g., output values fromlayers or neurons within a neural network) stored in an activationstorage 620 that are functions of input/output and/or weight parameterdata stored in code and/or data storage 601 and/or code and/or datastorage 605. In at least one embodiment, activations stored inactivation storage 620 are generated according to linear algebraic andor matrix-based mathematics performed by ALU(s) 610 in response toperforming instructions or other code, wherein weight values stored incode and/or data storage 605 and/or code and/or data storage 601 areused as operands along with other values, such as bias values, gradientinformation, momentum values, or other parameters or hyperparameters,any or all of which may be stored in code and/or data storage 605 orcode and/or data storage 601 or another storage on or off-chip.

In at least one embodiment, ALU(s) 610 are included within one or moreprocessors or other hardware logic devices or circuits, whereas inanother embodiment, ALU(s) 610 may be external to a processor or otherhardware logic device or circuit that uses them (e.g., a co-processor).In at least one embodiment, ALUs 610 may be included within aprocessor's execution units or otherwise within a bank of ALUsaccessible by a processor's execution units either within same processoror distributed between different processors of different types (e.g.,central processing units, graphics processing units, fixed functionunits, etc.). In at least one embodiment, code and/or data storage 601,code and/or data storage 605, and activation storage 620 may be on sameprocessor or other hardware logic device or circuit, whereas in anotherembodiment, they may be in different processors or other hardware logicdevices or circuits, or some combination of same and differentprocessors or other hardware logic devices or circuits. In at least oneembodiment, any portion of activation storage 620 may be included withother on-chip or off-chip data storage, including a processor's L1, L2,or L3 cache or system memory. Furthermore, inferencing and/or trainingcode may be stored with other code accessible to a processor or otherhardware logic or circuit and fetched and/or processed using aprocessor's fetch, decode, scheduling, execution, retirement and/orother logical circuits.

In at least one embodiment, activation storage 620 may be cache memory,DRAM, SRAM, non-volatile memory (e.g., Flash memory), or other storage.In at least one embodiment, activation storage 620 may be completely orpartially within or external to one or more processors or other logicalcircuits. In at least one embodiment, choice of whether activationstorage 620 is internal or external to a processor, for example, orincluded of DRAM, SRAM, Flash or some other storage type may depend onavailable storage on-chip versus off-chip, latency requirements oftraining and/or inferencing functions being performed, batch size ofdata used in inferencing and/or training of a neural network, or somecombination of these factors. In at least one embodiment, inferenceand/or training logic 615 illustrated in FIG. 6B may be used inconjunction with an application-specific integrated circuit (“ASIC”),such as Tensorflow® Processing Unit from Google, an inference processingunit (IPU) from Graphcore™, or a Nervana® (e.g., “Lake Crest”) processorfrom Intel Corp. In at least one embodiment, inference and/or traininglogic 615 illustrated in FIG. 6B may be used in conjunction with centralprocessing unit (“CPU”) hardware, graphics processing unit (“GPU”)hardware or other hardware, such as field programmable gate arrays(“FPGAs”).

In at least one embodiment, as illustrated in FIG. 6C, inference and/ortraining logic 615 may include, without limitation, hardware logic inwhich computational resources are dedicated or otherwise exclusivelyused in conjunction with weight values or other informationcorresponding to one or more layers of neurons within a neural network.In at least one embodiment, inference and/or training logic 615illustrated in FIG. 6C may be used in conjunction with anapplication-specific integrated circuit (ASIC), such as Tensorflow®Processing Unit from Google, an inference processing unit (IPU) fromGraphcore™, or a Nervana® (e.g., “Lake Crest”) processor from IntelCorp. In at least one embodiment, inference and/or training logic 615illustrated in FIG. 6C may be used in conjunction with centralprocessing unit (CPU) hardware, graphics processing unit (GPU) hardwareor other hardware, such as field programmable gate arrays (FPGAs). In atleast one embodiment, inference and/or training logic 615 includes,without limitation, code and/or data storage 601 and code and/or datastorage 605, which may be used to store code (e.g., graph code), weightvalues and/or other information, including bias values, gradientinformation, momentum values, and/or other parameter or hyperparameterinformation. In at least one embodiment illustrated in FIG. 6C, each ofcode and/or data storage 601 and code and/or data storage 605 isassociated with a dedicated computational resource, such ascomputational hardware 602 and computational hardware 606, respectively.

In at least one embodiment, each of code and/or data storage 601 and 605and corresponding computational hardware 602 and 606, respectively,correspond to different layers of a neural network, such that resultingactivation from one “storage/computational pair 601/602” of code and/ordata storage 601 and computational hardware 602 is provided as an inputto “storage/computational pair 605/606” of code and/or data storage 605and computational hardware 606, in order to mirror conceptualorganization of a neural network. In at least one embodiment, each ofstorage/computational pairs 601/602 and 605/606 may correspond to morethan one neural network layer. In at least one embodiment, additionalstorage/computation pairs (not shown) subsequent to or in parallel withstorage computation pairs 601/602 and 605/606 may be included ininference and/or training logic 615.

Computer Systems

FIG. 7A is a block diagram illustrating an exemplary computer system700A, which may be a system with interconnected devices and components,a system-on-a-chip (SOC) or some combination thereof formed with aprocessor that may include execution units to execute an instruction tosupport and/or to enable the intelligent control of therefrigerant-assisted cooling described herein, according to at least oneembodiment. In at least one embodiment, computer system 700A mayinclude, without limitation, a component, such as a processor 702 toemploy execution units including logic to perform algorithms for processdata, in accordance with present disclosure, such as in embodimentdescribed herein. In at least one embodiment, computer system 700A mayinclude processors, such as PENTIUM® Processor family, Xeon™, Itanium®,XScale™ and/or StrongARM™, Intel® Core™, or Intel® Nervana™microprocessors available from Intel Corporation of Santa Clara, Calif.,although other systems (including PCs having other microprocessors,engineering workstations, set-top boxes and like) may also be used. Inat least one embodiment, computer system 700B may execute a version ofWINDOWS' operating system available from Microsoft Corporation ofRedmond, Wash., although other operating systems (UNIX and Linux forexample), embedded software, and/or graphical user interfaces, may alsobe used.

In at least one embodiment, the exemplary computer system 700A mayincorporate one or more of components 110-116 (from FIG. 1) to supportprocessing aspects for the intelligent control for the multiple modecooling subsystem. For at least this reason, in one embodiment, FIG. 7Aillustrates a system, which includes interconnected hardware devices or“chips”, whereas in other embodiments, FIG. 7A may illustrate anexemplary System on a Chip (“SoC”). In at least one embodiment, devicesmay be interconnected with proprietary interconnects, standardizedinterconnects (e.g., PCIe) or some combination thereof. In at least oneembodiment, one or more components of computer system 700B areinterconnected using compute express link (CXL) interconnects. Inferenceand/or training logic 615 are used to perform inferencing and/ortraining operations associated with one or more embodiments, aspreviously discussed with respect to FIGS. 6A-C, for instance. Detailsregarding inference and/or training logic 615 are provided below inconjunction with FIGS. 6A-C. In at least one embodiment, inferenceand/or training logic 615 may be used in system FIG. 7A for inferencingor predicting operations based, at least in part, on weight parameterscalculated using neural network training operations, neural networkfunctions and/or architectures, or neural network use cases describedherein.

Embodiments may be used in other devices such as handheld devices andembedded applications. Some examples of handheld devices includecellular phones, Internet Protocol devices, digital cameras, personaldigital assistants (“PDAs”), and handheld PCs. In at least oneembodiment, embedded applications may include a microcontroller, adigital signal processor (“DSP”), system on a chip, network computers(“NetPCs”), set-top boxes, network hubs, wide area network (“WAN”)switches, or any other system that may perform one or more instructionsin accordance with at least one embodiment.

In at least one embodiment, computer system 700A may include, withoutlimitation, processor 702 that may include, without limitation, one ormore execution units 708 to perform machine learning model trainingand/or inferencing according to techniques described herein. In at leastone embodiment, computer system 700A is a single processor desktop orserver system, but in another embodiment computer system 700A may be amultiprocessor system. In at least one embodiment, processor 702 mayinclude, without limitation, a complex instruction set computer (“CISC”)microprocessor, a reduced instruction set computing (“RISC”)microprocessor, a very long instruction word (“VLIW”) microprocessor, aprocessor implementing a combination of instruction sets, or any otherprocessor device, such as a digital signal processor, for example. In atleast one embodiment, processor 702 may be coupled to a processor bus710 that may transmit data signals between processor 702 and othercomponents in computer system 700A.

In at least one embodiment, processor 702 may include, withoutlimitation, a Level 1 (“L1”) internal cache memory (“cache”) 704. In atleast one embodiment, processor 702 may have a single internal cache ormultiple levels of internal cache. In at least one embodiment, cachememory may reside external to processor 702. Other embodiments may alsoinclude a combination of both internal and external caches depending onparticular implementation and needs. In at least one embodiment,register file 706 may store different types of data in various registersincluding, without limitation, integer registers, floating pointregisters, status registers, and instruction pointer register.

In at least one embodiment, execution unit 708, including, withoutlimitation, logic to perform integer and floating point operations, alsoresides in processor 702. In at least one embodiment, processor 702 mayalso include a microcode (“ucode”) read only memory (“ROM”) that storesmicrocode for certain macro instructions. In at least one embodiment,execution unit 708 may include logic to handle a packed instruction set709. In at least one embodiment, by including packed instruction set 709in an instruction set of a general-purpose processor 702, along withassociated circuitry to execute instructions, operations used by manymultimedia applications may be performed using packed data in ageneral-purpose processor 702. In one or more embodiments, manymultimedia applications may be accelerated and executed more efficientlyby using full width of a processor's data bus for performing operationson packed data, which may eliminate need to transfer smaller units ofdata across processor's data bus to perform one or more operations onedata element at a time.

In at least one embodiment, execution unit 708 may also be used inmicrocontrollers, embedded processors, graphics devices, DSPs, and othertypes of logic circuits. In at least one embodiment, computer system700A may include, without limitation, a memory 720. In at least oneembodiment, memory 720 may be implemented as a Dynamic Random AccessMemory (“DRAM”) device, a Static Random Access Memory (“SRAM”) device,flash memory device, or other memory device. In at least one embodiment,memory 720 may store instruction(s) 719 and/or data 721 represented bydata signals that may be executed by processor 702.

In at least one embodiment, system logic chip may be coupled toprocessor bus 710 and memory 720. In at least one embodiment, systemlogic chip may include, without limitation, a memory controller hub(“MCH”) 716, and processor 702 may communicate with MCH 716 viaprocessor bus 710. In at least one embodiment, MCH 716 may provide ahigh bandwidth memory path 718 to memory 720 for instruction and datastorage and for storage of graphics commands, data and textures. In atleast one embodiment, MCH 716 may direct data signals between processor702, memory 720, and other components in computer system 700A and tobridge data signals between processor bus 710, memory 720, and a systemI/O 722. In at least one embodiment, system logic chip may provide agraphics port for coupling to a graphics controller. In at least oneembodiment, MCH 716 may be coupled to memory 720 through a highbandwidth memory path 718 and graphics/video card 712 may be coupled toMCH 716 through an Accelerated Graphics Port (“AGP”) interconnect 714.

In at least one embodiment, computer system 700A may use system I/O 722that is a proprietary hub interface bus to couple MCH 716 to I/Ocontroller hub (“ICH”) 730. In at least one embodiment, ICH 730 mayprovide direct connections to some I/O devices via a local I/O bus. Inat least one embodiment, local I/O bus may include, without limitation,a high-speed I/O bus for connecting peripherals to memory 720, chipset,and processor 702. Examples may include, without limitation, an audiocontroller 729, a firmware hub (“flash BIOS”) 728, a wirelesstransceiver 726, a data storage 724, a legacy I/O controller 723containing user input and keyboard interfaces 725, a serial expansionport 727, such as Universal Serial Bus (“USB”), and a network controller734. Data storage 724 may include a hard disk drive, a floppy diskdrive, a CD-ROM device, a flash memory device, or other mass storagedevice.

FIG. 7B is a block diagram illustrating an electronic device 700B forutilizing a processor 710 to support and/or to enable intelligentcontrol of the multiple mode cooling subsystem described herein,according to at least one embodiment. In at least one embodiment,electronic device 700B may be, for example and without limitation, anotebook, a tower server, a rack server, a blade server, a laptop, adesktop, a tablet, a mobile device, a phone, an embedded computer, orany other suitable electronic device. In at least one embodiment, theexemplary electronic device 700B may incorporate one or more ofcomponents that support processing aspects for the multiple mode coolingsubsystem.

In at least one embodiment, system 700B may include, without limitation,processor 710 communicatively coupled to any suitable number or kind ofcomponents, peripherals, modules, or devices. In at least oneembodiment, processor 710 coupled using a bus or interface, such as a 1°C. bus, a System Management Bus (“SMBus”), a Low Pin Count (LPC) bus, aSerial Peripheral Interface (“SPI”), a High Definition Audio (“HDA”)bus, a Serial Advance Technology Attachment (“SATA”) bus, a UniversalSerial Bus (“USB”) (versions 1, 2, 3), or a Universal AsynchronousReceiver/Transmitter (“UART”) bus. In at least one embodiment, FIG. 7Billustrates a system, which includes interconnected hardware devices or“chips”, whereas in other embodiments, FIG. 7B may illustrate anexemplary System on a Chip (“SoC”). In at least one embodiment, devicesillustrated in FIG. 7B may be interconnected with proprietaryinterconnects, standardized interconnects (e.g., PCIe) or somecombination thereof. In at least one embodiment, one or more componentsof FIG. 7B are interconnected using compute express link (CXL)interconnects.

In at least one embodiment, FIG. 7B may include a display 724, a touchscreen 725, a touch pad 730, a Near Field Communications unit (“NFC”)745, a sensor hub 740, a thermal sensor 746, an Express Chipset (“EC”)735, a Trusted Platform Module (“TPM”) 738, BIOS/firmware/flash memory(“BIOS, FW Flash”) 722, a DSP 760, a drive 720 such as a Solid StateDisk (“SSD”) or a Hard Disk Drive (“HDD”), a wireless local area networkunit (“WLAN”) 750, a Bluetooth unit 752, a Wireless Wide Area Networkunit (“WWAN”) 756, a Global Positioning System (GPS) 755, a camera (“USB3.0 camera”) 754 such as a USB 3.0 camera, and/or a Low Power DoubleData Rate (“LPDDR”) memory unit (“LPDDR3”) 715 implemented in, forexample, LPDDR3 standard. These components may each be implemented inany suitable manner.

In at least one embodiment, other components may be communicativelycoupled to processor 710 through components discussed above. In at leastone embodiment, an accelerometer 741, Ambient Light Sensor (“ALS”) 742,compass 743, and a gyroscope 744 may be communicatively coupled tosensor hub 740. In at least one embodiment, thermal sensor 739, a fan737, a keyboard 746, and a touch pad 730 may be communicatively coupledto EC 735. In at least one embodiment, speaker 763, headphones 764, andmicrophone (“mic”) 765 may be communicatively coupled to an audio unit(“audio codec and class d amp”) 762, which may in turn becommunicatively coupled to DSP 760. In at least one embodiment, audiounit 764 may include, for example and without limitation, an audiocoder/decoder (“codec”) and a class D amplifier. In at least oneembodiment, SIM card (“SIM”) 757 may be communicatively coupled to WWANunit 756. In at least one embodiment, components such as WLAN unit 750and Bluetooth unit 752, as well as WWAN unit 756 may be implemented in aNext Generation Form Factor (“NGFF”).

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,inference and/or training logic x615 may be used in system FIG. 7B forinferencing or predicting operations based, at least in part, on weightparameters calculated using neural network training operations, neuralnetwork functions and/or architectures, or neural network use casesdescribed herein.

FIG. 7C illustrates a computer system 700C, according to at least oneembodiment, to support and/or to enable the intelligent control of themultiple mode cooling subsystem described herein. In at least oneembodiment, computer system 700C includes, without limitation, acomputer 771 and a USB stick 770. In at least one embodiment, computer771 may include, without limitation, any number and type of processor(s)(not shown) and a memory (not shown). In at least one embodiment,computer 771 includes, without limitation, a server, a cloud instance, alaptop, and a desktop computer.

In at least one embodiment, USB stick 770 includes, without limitation,a processing unit 772, a USB interface 774, and USB interface logic 773.In at least one embodiment, processing unit 772 may be any instructionexecution system, apparatus, or device capable of executinginstructions. In at least one embodiment, processing unit 772 mayinclude, without limitation, any number and type of processing cores(not shown). In at least one embodiment, processing unit or core 772comprises an application specific integrated circuit (“ASIC”) that isoptimized to perform any amount and type of operations associated withmachine learning. For instance, in at least one embodiment, processingcore 772 is a tensor processing unit (“TPC”) that is optimized toperform machine learning inference operations. In at least oneembodiment, processing core 772 is a vision processing unit (“VPU”) thatis optimized to perform machine vision and machine learning inferenceoperations.

In at least one embodiment, USB interface 774 may be any type of USBconnector or USB socket. For instance, in at least one embodiment, USBinterface 774 is a USB 3.0 Type-C socket for data and power. In at leastone embodiment, USB interface 774 is a USB 3.0 Type-A connector. In atleast one embodiment, USB interface logic 773 may include any amount andtype of logic that enables processing unit 772 to interface with ordevices (e.g., computer 771) via USB connector 774.

Inference and/or training logic 615, as described with respect to FIGS.6B and 6C, are used to perform inferencing and/or training operationsassociated with one or more embodiments. Details regarding inferenceand/or training logic 615 are provided below in conjunction with FIGS.6B and/or 6C. In at least one embodiment, inference and/or traininglogic 615 may be used in system FIG. 7C for inferencing or predictingoperations based, at least in part, on weight parameters calculatedusing neural network training operations, neural network functionsand/or architectures, or neural network use cases described herein.

FIG. 8 illustrates a further example computer system 800, according toat least one embodiment, to implement various processes and methods forthe multiple mode cooling subsystem described throughout thisdisclosure. In at least one embodiment, computer system 800 includes,without limitation, at least one central processing unit (“CPU”) 802that is connected to a communication bus 810 implemented using anysuitable protocol, such as PCI (“Peripheral Component Interconnect”),peripheral component interconnect express (“PCI-Express”), AGP(“Accelerated Graphics Port”), HyperTransport, or any other bus orpoint-to-point communication protocol(s). In at least one embodiment,computer system 800 includes, without limitation, a main memory 804 andcontrol logic (e.g., implemented as hardware, software, or a combinationthereof) and data are stored in main memory 804 which may take form ofrandom access memory (“RAM”). In at least one embodiment, a networkinterface subsystem (“network interface”) 822 provides an interface toother computing devices and networks for receiving data from andtransmitting data to other systems from computer system 800.

In at least one embodiment, computer system 800, in at least oneembodiment, includes, without limitation, input devices 808, parallelprocessing system 812, and display devices 806 which can be implementedusing a cathode ray tube (“CRT”), liquid crystal display (“LCD”), lightemitting diode (“LED”), plasma display, or other suitable displaytechnologies. In at least one embodiment, user input is received frominput devices 808 such as keyboard, mouse, touchpad, microphone, andmore. In at least one embodiment, each of foregoing modules can besituated on a single semiconductor platform to form a processing system.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments, aspreviously discussed with respect to FIGS. 6A-C, for instance. Detailsregarding inference and/or training logic 615 are provided below inconjunction with FIGS. 6A-C. In at least one embodiment, inferenceand/or training logic 615 may be used in system FIG. 8 for inferencingor predicting operations based, at least in part, on weight parameterscalculated using neural network training operations, neural networkfunctions and/or architectures, or neural network use cases describedherein. In at least one embodiment, inference and/or training logic 615may be used in system FIG. 8 for inferencing or predicting operationsbased, at least in part, on weight parameters calculated using neuralnetwork training operations, neural network functions and/orarchitectures, or neural network use cases described herein.

FIG. 9A illustrates an exemplary architecture in which a plurality ofGPUs 910-913 is communicatively coupled to a plurality of multi-coreprocessors 905-906 over high-speed links 940-943 (e.g., buses,point-to-point interconnects, etc.). In one embodiment, high-speed links940-943 support a communication throughput of 4 GB/s, 30 GB/s, 80 GB/sor higher. Various interconnect protocols may be used including, but notlimited to, PCIe 4.0 or 5.0 and NVLink 2.0.

In addition, and in one embodiment, two or more of GPUs 910-913 areinterconnected over high-speed links 929-930, which may be implementedusing same or different protocols/links than those used for high-speedlinks 940-943. Similarly, two or more of multi-core processors 905-906may be connected over high speed link 928 which may be symmetricmulti-processor (SMP) buses operating at 20 GB/s, 30 GB/s, 120 GB/s orhigher. Alternatively, all communication between various systemcomponents shown in FIG. 9A may be accomplished using sameprotocols/links (e.g., over a common interconnection fabric).

In one embodiment, each multi-core processor 905-906 is communicativelycoupled to a processor memory 901-902, via memory interconnects 926-927,respectively, and each GPU 910-913 is communicatively coupled to GPUmemory 920-923 over GPU memory interconnects 950-953, respectively.Memory interconnects 926-927 and 950-953 may utilize same or differentmemory access technologies. By way of example, and not limitation,processor memories 901-902 and GPU memories 920-923 may be volatilememories such as dynamic random access memories (DRAMs) (includingstacked DRAMs), Graphics DDR SDRAM (GDDR) (e.g., GDDR5, GDDR6), or HighBandwidth Memory (HBM) and/or may be non-volatile memories such as 3DXPoint or Nano-Ram. In one embodiment, some portion of processormemories 901-902 may be volatile memory and another portion may benon-volatile memory (e.g., using a two-level memory (2LM) hierarchy).

As described below, although various processors 905-906 and GPUs 910-913may be physically coupled to a particular memory 901-902, 920-923,respectively, a unified memory architecture may be implemented in whicha same virtual system address space (also referred to as “effectiveaddress” space) is distributed among various physical memories. In atleast one embodiment, processor memories 901-902 may each include 64 GBof system memory address space and GPU memories 920-923 may each include32 GB of system memory address space (resulting in a total of 256 GBaddressable memory in this example).

As discussed elsewhere in this disclosure, at least flow rates andassociated temperatures may be established for a first level of anintelligent learning system, such as a neural network system. As thefirst level represents the prior data, it also represents a smallersubset of the data that may be available to improve the system byretraining the system. The testing and training may be performed inparallel using the multiple processor units so that the intelligentlearning system is robust. An architecture, such as in FIG. 9A, may beused. When convergence is achieved for the intelligent learning system,an amount of data points and the data in the data points used to causethe convergence is noted. The data and data points may be used tocontrol the multiple mode cooling subsystem as discussed in reference,for instance, to FIGS. 2-5.

FIG. 9B illustrates additional details for an interconnection between amulti-core processor 907 and a graphics acceleration module 946 inaccordance with one exemplary embodiment. Graphics acceleration module946 may include one or more GPU chips integrated on a line card which iscoupled to processor 907 via high-speed link 940. Alternatively,graphics acceleration module 946 may be integrated on a same package orchip as processor 907.

In at least one embodiment, illustrated processor 907 includes aplurality of cores 960A-960D, each with a translation lookaside buffer961A-961D and one or more caches 962A-962D. In at least one embodiment,cores 960A-960D may include various other components for executinginstructions and processing data which are not illustrated. Caches962A-962D may include level 1 (L1) and level 2 (L2) caches. In addition,one or more shared caches 956 may be included in caches 962A-962D andshared by sets of cores 960A-960D. In at least one embodiment, oneembodiment of processor 907 includes 24 cores, each with its own L1cache, twelve shared L2 caches, and twelve shared L3 caches. In thisembodiment, one or more L2 and L3 caches are shared by two adjacentcores. Processor 907 and graphics acceleration module 946 connect withsystem memory 914, which may include processor memories 901-902 of FIG.9A.

Coherency is maintained for data and instructions stored in variouscaches 962A-962D, 956 and system memory 914 via inter-core communicationover a coherence bus 964. In at least one embodiment, each cache mayhave cache coherency logic/circuitry associated therewith to communicateto over coherence bus 964 in response to detected reads or writes toparticular cache lines. In one implementation, a cache snooping protocolis implemented over coherence bus 964 to snoop cache accesses.

In one embodiment, a proxy circuit 925 communicatively couples graphicsacceleration module 946 to coherence bus 964, allowing graphicsacceleration module 946 to participate in a cache coherence protocol asa peer of cores 960A-960D. In particular, an interface 935 providesconnectivity to proxy circuit 925 over high-speed link 940 (e.g., a PCIebus, NVLink, etc.) and an interface 937 connects graphics accelerationmodule 946 to link 940.

In one implementation, an accelerator integration circuit 936 providescache management, memory access, context management, and interruptmanagement services on behalf of a plurality of graphics processingengines 931, 932, N of graphics acceleration module 946. Graphicsprocessing engines 931, 932, N may each include a separate graphicsprocessing unit (GPU). Alternatively, graphics processing engines 931,932, N may include different types of graphics processing engines withina GPU such as graphics execution units, media processing engines (e.g.,video encoders/decoders), samplers, and blit engines. In at least oneembodiment, graphics acceleration module 946 may be a GPU with aplurality of graphics processing engines 931-932, N or graphicsprocessing engines 931-932, N may be individual GPUs integrated on acommon package, line card, or chip. As is the case, the abovedetermination for the reconstruction parameter and the reconstructionalgorithm may be performed in GPUs 931-N of FIG. 9B.

In one embodiment, accelerator integration circuit 936 includes a memorymanagement unit (MMU) 939 for performing various memory managementfunctions such as virtual-to-physical memory translations (also referredto as effective-to-real memory translations) and memory access protocolsfor accessing system memory 914. MMU 939 may also include a translationlookaside buffer (TLB) (not shown) for caching virtual/effective tophysical/real address translations. In one implementation, a cache 938stores commands and data for efficient access by graphics processingengines 931-932, N. In one embodiment, data stored in cache 938 andgraphics memories 933-934, M is kept coherent with core caches962A-962D, 956, and system memory 914. As mentioned above, this may beaccomplished via proxy circuit 925 on behalf of cache 938 and memories933-934, M (e.g., sending updates to cache 938 related tomodifications/accesses of cache lines on processor caches 962A-962D,956, and receiving updates from cache 938).

A set of registers 945 store context data for threads executed bygraphics processing engines 931-932, N and a context management circuit948 manages thread contexts. In at least one embodiment, contextmanagement circuit 948 may perform save and restore operations to saveand restore contexts of various threads during contexts switches (e.g.,where a first thread is saved and a second thread is stored so that asecond thread can be executed by a graphics processing engine). In atleast one embodiment, on a context switch, context management circuit948 may store current register values to a designated region in memory(e.g., identified by a context pointer). It may then restore registervalues when returning to a context. In one embodiment, an interruptmanagement circuit 947 receives and processes interrupts received fromsystem devices.

In one implementation, virtual/effective addresses from a graphicsprocessing engine 931 are translated to real/physical addresses insystem memory 914 by MMU 939. One embodiment of accelerator integrationcircuit 936 supports multiple (e.g., 4, 8, 16) graphics acceleratormodules 946 and/or other accelerator devices. Graphics acceleratormodule 946 may be dedicated to a single application executed onprocessor 907 or may be shared between multiple applications. In oneembodiment, a virtualized graphics execution environment is presented inwhich resources of graphics processing engines 931-932, N are sharedwith multiple applications or virtual machines (VMs). In at least oneembodiment, resources may be subdivided into “slices” which areallocated to different VMs and/or applications based on processingrequirements and priorities associated with VMs and/or applications.

In at least one embodiment, accelerator integration circuit 936 performsas a bridge to a system for graphics acceleration module 946 andprovides address translation and system memory cache services. Inaddition, accelerator integration circuit 936 may provide virtualizationfacilities for a host processor to manage virtualization of graphicsprocessing engines 931-932, N, interrupts, and memory management.

Because hardware resources of graphics processing engines 931-932, N aremapped explicitly to a real address space seen by host processor 907,any host processor can address these resources directly using aneffective address value. One function of accelerator integration circuit936, in one embodiment, is physical separation of graphics processingengines 931-932, N so that they appear to a system as independent units.

In at least one embodiment, one or more graphics memories 933-934, M arecoupled to each of graphics processing engines 931-932, N, respectively.Graphics memories 933-934, M store instructions and data being processedby each of graphics processing engines 931-932, N. Graphics memories933-934, M may be volatile memories such as DRAMs (including stackedDRAMs), GDDR memory (e.g., GDDR5, GDDR6), or HBM, and/or may benon-volatile memories such as 3D XPoint or Nano-Ram.

In one embodiment, to reduce data traffic over link 940, biasingtechniques are used to ensure that data stored in graphics memories933-934, M is data which will be used most frequently by graphicsprocessing engines 931-932, N and may not used by cores 960A-960D (atleast not frequently). Similarly, a biasing mechanism attempts to keepdata needed by cores (and may not graphics processing engines 931-932,N) within caches 962A-962D, 956 of cores and system memory 914.

FIG. 9C illustrates another exemplary embodiment in which acceleratorintegration circuit 936 is integrated within processor 907 for enablingand/or supporting intelligent control of the multiple mode coolingsubsystem, according to at least one embodiment of the disclosureherein. In at least this embodiment, graphics processing engines931-932, N communicate directly over high-speed link 940 to acceleratorintegration circuit 936 via interface 937 and interface 935 (which,again, may be utilize any form of bus or interface protocol).Accelerator integration circuit 936 may perform same operations as thosedescribed with respect to FIG. 9B, but potentially at a higherthroughput given its close proximity to coherence bus 964 and caches962A-962D, 956. At least one embodiment supports different programmingmodels including a dedicated-process programming model (no graphicsacceleration module virtualization) and shared programming models (withvirtualization), which may include programming models which arecontrolled by accelerator integration circuit 936 and programming modelswhich are controlled by graphics acceleration module 946.

In at least one embodiment, graphics processing engines 931-932, N arededicated to a single application or process under a single operatingsystem. In at least one embodiment, a single application can funnelother application requests to graphics processing engines 931-932, N,providing virtualization within a VM/partition.

In at least one embodiment, graphics processing engines 931-932, N, maybe shared by multiple VM/application partitions. In at least oneembodiment, shared models may use a system hypervisor to virtualizegraphics processing engines 931-932, N to allow access by each operatingsystem. For single-partition systems without a hypervisor, graphicsprocessing engines 931-932, N are owned by an operating system. In atleast one embodiment, an operating system can virtualize graphicsprocessing engines 931-932, N to provide access to each process orapplication.

In at least one embodiment, graphics acceleration module 946 or anindividual graphics processing engine 931-932, N selects a processelement using a process handle. In at least one embodiment, processelements are stored in system memory 914 and are addressable using aneffective address to real address translation techniques describedherein. In at least one embodiment, a process handle may be animplementation-specific value provided to a host process whenregistering its context with graphics processing engine 931-932, N (thatis, calling system software to add a process element to a processelement linked list). In at least one embodiment, a lower 16-bits of aprocess handle may be an offset of a process element within a processelement linked list.

FIG. 9D illustrates an exemplary accelerator integration slice 990 forenabling and/or supporting intelligent control of the multiple modecooling subsystem, according to at least one embodiment of thedisclosure herein. As used herein, a “slice” comprises a specifiedportion of processing resources of accelerator integration circuit 936.Application effective address space 982 within system memory 914 storesprocess elements 983. In one embodiment, process elements 983 are storedin response to GPU invocations 981 from applications 980 executed onprocessor 907. A process element 983 contains process state forcorresponding application 980. A work descriptor (WD) 984 contained inprocess element 983 can be a single job requested by an application ormay contain a pointer to a queue of jobs. In at least one embodiment, WD984 is a pointer to a job request queue in an application's addressspace 982.

Graphics acceleration module 946 and/or individual graphics processingengines 931-932, N can be shared by all or a subset of processes in asystem. In at least one embodiment, an infrastructure for setting upprocess state and sending a WD 984 to a graphics acceleration module 946to start a job in a virtualized environment may be included.

In at least one embodiment, a dedicated-process programming model isimplementation-specific. In this model, a single process owns graphicsacceleration module 946 or an individual graphics processing engine 931.Because graphics acceleration module 946 is owned by a single process, ahypervisor initializes accelerator integration circuit 936 for an owningpartition and an operating system initializes accelerator integrationcircuit 936 for an owning process when graphics acceleration module 946is assigned.

In operation, a WD fetch unit 991 in accelerator integration slice 990fetches next WD 984 which includes an indication of work to be done byone or more graphics processing engines of graphics acceleration module946. Data from WD 984 may be stored in registers 945 and used by MMU939, interrupt management circuit 947, and/or context management circuit948 as illustrated. In at least one embodiment, one embodiment of MMU939 includes segment/page walk circuitry for accessing segment/pagetables 986 within OS virtual address space 985. Interrupt managementcircuit 947 may process interrupt events 992 received from graphicsacceleration module 946. When performing graphics operations, aneffective address 993 generated by a graphics processing engine 931-932,N is translated to a real address by MMU 939.

In one embodiment, a same set of registers 945 are duplicated for eachgraphics processing engine 931-932, N and/or graphics accelerationmodule 946 and may be initialized by a hypervisor or operating system.Each of these duplicated registers may be included in an acceleratorintegration slice 990. Exemplary registers that may be initialized by ahypervisor are shown in Table 1.

TABLE 1 Hypervisor Initialized Registers 1 Slice Control Register 2 RealAddress (RA) Scheduled Processes Area Pointer 3 Authority Mask OverrideRegister 4 Interrupt Vector Table Entry Offset 5 Interrupt Vector TableEntry Limit 6 State Register 7 Logical Partition ID 8 Real address (RA)Hypervisor Accelerator Utilization Record Pointer 9 Storage DescriptionRegister

Exemplary registers that may be initialized by an operating system areshown in Table 2.

TABLE 2 Operating System Initialized Registers 1 Process and ThreadIdentification 2 Effective Address (EA) Context Save/Restore Pointer 3Virtual Address (VA) Accelerator Utilization Record Pointer 4 VirtualAddress (VA) Storage Segment Table Pointer 5 Authority Mask 6 Workdescriptor

In one embodiment, each WD 984 is specific to a particular graphicsacceleration module 946 and/or graphics processing engines 931-932, N.It contains all information required by a graphics processing engine931-932, N to do work or it can be a pointer to a memory location wherean application has set up a command queue of work to be completed.

FIG. 9E illustrates additional details for one exemplary embodiment of ashared model. This embodiment includes a hypervisor real address space998 in which a process element list 999 is stored. Hypervisor realaddress space 998 is accessible via a hypervisor 996 which virtualizesgraphics acceleration module engines for operating system 995.

In at least one embodiment, shared programming models allow for all or asubset of processes from all or a subset of partitions in a system touse a graphics acceleration module 946. There are two programming modelswhere graphics acceleration module 946 is shared by multiple processesand partitions: time-sliced shared and graphics-directed shared.

In this model, system hypervisor 996 owns graphics acceleration module946 and makes its function available to all operating systems 995. For agraphics acceleration module 946 to support virtualization by systemhypervisor 996, graphics acceleration module 946 may adhere to thefollowing: 1) An application's job request must be autonomous (that is,state does not need to be maintained between jobs), or graphicsacceleration module 946 must provide a context save and restoremechanism. 2) An application's job request is guaranteed by graphicsacceleration module 946 to complete in a specified amount of time,including any translation faults, or graphics acceleration module 946provides an ability to preempt processing of a job. 3) Graphicsacceleration module 946 must be guaranteed fairness between processeswhen operating in a directed shared programming model.

In at least one embodiment, application 980 is required to make anoperating system 995 system call with a graphics acceleration module 946type, a work descriptor (WD), an authority mask register (AMR) value,and a context save/restore area pointer (CSRP). In at least oneembodiment, graphics acceleration module 946 type describes a targetedacceleration function for a system call. In at least one embodiment,graphics acceleration module 946 type may be a system-specific value. Inat least one embodiment, WD is formatted specifically for graphicsacceleration module 946 and can be in a form of a graphics accelerationmodule 946 command, an effective address pointer to a user-definedstructure, an effective address pointer to a queue of commands, or anyother data structure to describe work to be done by graphicsacceleration module 946. In one embodiment, an AMR value is an AMR stateto use for a current process. In at least one embodiment, a value passedto an operating system is similar to an application setting an AMR. Ifaccelerator integration circuit 936 and graphics acceleration module 946implementations do not support a User Authority Mask Override Register(UAMOR), an operating system may apply a current UAMOR value to an AMRvalue before passing an AMR in a hypervisor call. Hypervisor 996 may, inat least one embodiment, apply a current Authority Mask OverrideRegister (AMOR) value before placing an AMR into process element 983. Inat least one embodiment, CSRP is one of registers 945 containing aneffective address of an area in an application's effective address space982 for graphics acceleration module 946 to save and restore contextstate. This pointer is used in at least one embodiment, if no state isrequired to be saved between jobs or when a job is preempted. In atleast one embodiment, context save/restore area may be pinned systemmemory.

Upon receiving a system call, operating system 995 may verify thatapplication 980 has registered and been given authority to use graphicsacceleration module 946. Operating system 995 then calls hypervisor 996with information shown in Table 3.

TABLE 3 OS to Hypervisor Call Parameters 1 A work descriptor (WD) 2 AnAuthority Mask Register (AMR) value (potentially masked). 3 An effectiveaddress (EA) Context Save/Restore Area Pointer (CSRP) 4 A process ID(PID) and optional thread ID (TID) 5 A virtual address (VA) acceleratorutilization record pointer (AURP) 6 The virtual address of the storagesegment table pointer (SSTP) 7 A logical interrupt service number (LISN)

Upon receiving a hypervisor call, hypervisor 996 verifies that operatingsystem 995 has registered and been given authority to use graphicsacceleration module 946. Hypervisor 996 then puts process element 983into a process element linked list for a corresponding graphicsacceleration module 946 type. A process element may include informationshown in Table 4.

TABLE 4 Process Element Information 1 A work descriptor (WD) 2 AnAuthority Mask Register (AMR) value (potentially masked). 3 An effectiveaddress (EA) Context Save/Restore Area Pointer (CSRP) 4 A process ID(PID) and optional thread ID (TID) 5 A virtual address (VA) acceleratorutilization record pointer (AURP) 6 The virtual address of the storagesegment table pointer (SSTP) 7 A logical interrupt service number (LISN)8 Interrupt vector table, derived from the hypervisor call parameters. 9A state register (SR) value 10 A logical partition ID (LPID) 11 A realaddress (RA) hypervisor accelerator utilization record pointer 12 TheStorage Descriptor Register (SDR)

In at least one embodiment, hypervisor initializes a plurality ofaccelerator integration slice 990 registers 945.

As illustrated in FIG. 9F, in at least one embodiment, a unified memoryis used, addressable via a common virtual memory address space used toaccess physical processor memories 901-902 and GPU memories 920-923. Inthis implementation, operations executed on GPUs 910-913 utilize a samevirtual/effective memory address space to access processor memories901-902 and vice versa, thereby simplifying programmability. In oneembodiment, a first portion of a virtual/effective address space isallocated to processor memory 901, a second portion to second processormemory 902, a third portion to GPU memory 920, and so on. In at leastone embodiment, an entire virtual/effective memory space (sometimesreferred to as an effective address space) is thereby distributed acrosseach of processor memories 901-902 and GPU memories 920-923, allowingany processor or GPU to access any physical memory with a virtualaddress mapped to that memory.

In one embodiment, bias/coherence management circuitry 994A-994E withinone or more of MMUs 939A-939E ensures cache coherence between caches ofone or more host processors (e.g., 905) and GPUs 910-913 and implementsbiasing techniques indicating physical memories in which certain typesof data should be stored. While multiple instances of bias/coherencemanagement circuitry 994A-994E are illustrated in FIG. 9F,bias/coherence circuitry may be implemented within an MMU of one or morehost processors 905 and/or within accelerator integration circuit 936.

One embodiment allows GPU-attached memory 920-923 to be mapped as partof system memory, and accessed using shared virtual memory (SVM)technology, but without suffering performance drawbacks associated withfull system cache coherence. In at least one embodiment, an ability forGPU-attached memory 920-923 to be accessed as system memory withoutonerous cache coherence overhead provides a beneficial operatingenvironment for GPU offload. This arrangement allows host processor 905software to setup operands and access computation results, withoutoverhead of tradition I/O DMA data copies. Such traditional copiesinvolve driver calls, interrupts and memory mapped I/O (MMIO) accessesthat are all inefficient relative to simple memory accesses. In at leastone embodiment, an ability to access GPU attached memory 920-923 withoutcache coherence overheads can be critical to execution time of anoffloaded computation. In cases with substantial streaming write memorytraffic, for example, cache coherence overhead can significantly reducean effective write bandwidth seen by a GPU 910-913. In at least oneembodiment, efficiency of operand setup, efficiency of results access,and efficiency of GPU computation may play a role in determiningeffectiveness of a GPU offload.

In at least one embodiment, selection of GPU bias and host processorbias is driven by a bias tracker data structure. A bias table may beused, for example, which may be a page-granular structure (in at leastone embodiment this may be controlled at a granularity of a memory page)that includes 1 or 2 bits per GPU-attached memory page. In at least oneembodiment, a bias table may be implemented in a stolen memory range ofone or more GPU-attached memories 920-923, with or without a bias cachein GPU 910-913 (e.g., to cache frequently/recently used entries of abias table). Alternatively, an entire bias table may be maintainedwithin a GPU.

In at least one embodiment, a bias table entry associated with eachaccess to GPU-attached memory 920-923 is accessed prior to actual accessto a GPU memory, causing the following operations. First, local requestsfrom GPU 910-913 that find their page in GPU bias are forwarded directlyto a corresponding GPU memory 920-923. Local requests from a GPU thatfind their page in host bias are forwarded to processor 905 (e.g., overa high-speed link as discussed above). In one embodiment, requests fromprocessor 905 that find a requested page in host processor bias completea request like a normal memory read. Alternatively, requests directed toa GPU-biased page may be forwarded to GPU 910-913. In at least oneembodiment, a GPU may then transition a page to a host processor bias ifit is not currently using a page. In at least one embodiment, bias stateof a page can be changed either by a software-based mechanism, ahardware-assisted software-based mechanism, or, for a limited set ofcases, a purely hardware-based mechanism.

One mechanism for changing bias state employs an API call (e.g.,OpenCL), which, in turn, calls a GPU's device driver which, in turn,sends a message (or enqueues a command descriptor) to a GPU directing itto change a bias state and, for some transitions, perform a cacheflushing operation in a host. In at least one embodiment, cache flushingoperation is used for a transition from host processor 905 bias to GPUbias, but is not for an opposite transition.

In one embodiment, cache coherency is maintained by temporarilyrendering GPU-biased pages uncacheable by host processor 905. To accessthese pages, processor 905 may request access from GPU 910 which may ormay not grant access right away. Thus, to reduce communication betweenprocessor 905 and GPU 910 it is beneficial to ensure that GPU-biasedpages are those which are required by a GPU but not host processor 905and vice versa.

Inference and/or training logic 615 are used to perform one or moreembodiments. Details regarding the inference and/or training logic 615are provided below in conjunction with FIGS. 6B and/or 6C.

FIG. 10A illustrates exemplary integrated circuits and associatedgraphics processors that may be fabricated using one or more IP cores,according to various embodiments described herein, to support and/or toenable the multiple mode cooling subsystem. In addition to what isillustrated, other logic and circuits may be included in at least oneembodiment, including additional graphics processors/cores, peripheralinterface controllers, or general-purpose processor cores.

FIG. 10A is a block diagram illustrating an exemplary system on a chipintegrated circuit 1000A that may be fabricated using one or more IPcores, according to at least one embodiment. In at least one embodiment,integrated circuit 1000A includes one or more application processor(s)1005 (e.g., CPUs), at least one graphics processor 1010, and mayadditionally include an image processor 1015 and/or a video processor1020, any of which may be a modular IP core. In at least one embodiment,integrated circuit 1000A includes peripheral or bus logic including aUSB controller 1025, UART controller 1030, an SPI/SDIO controller 1035,and an I²S/I²C controller 1040. In at least one embodiment, integratedcircuit 1000A can include a display device 1045 coupled to one or moreof a high-definition multimedia interface (HDMI) controller 1050 and amobile industry processor interface (MIPI) display interface 1055. In atleast one embodiment, storage may be provided by a flash memorysubsystem 1060 including flash memory and a flash memory controller. Inat least one embodiment, memory interface may be provided via a memorycontroller 1065 for access to SDRAM or SRAM memory devices. In at leastone embodiment, some integrated circuits additionally include anembedded security engine 1070.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,inference and/or training logic 615 may be used in integrated circuit1000A for inferencing or predicting operations based, at least in part,on weight parameters calculated using neural network trainingoperations, neural network functions and/or architectures, or neuralnetwork use cases described herein.

FIGS. 10B-10C illustrate exemplary integrated circuits and associatedgraphics processors that may be fabricated using one or more IP cores,according to various embodiments described herein to support and/or toenable the multiple mode cooling subsystem. In addition to what isillustrated, other logic and circuits may be included in at least oneembodiment, including additional graphics processors/cores, peripheralinterface controllers, or general-purpose processor cores.

FIGS. 10B-10C are block diagrams illustrating exemplary graphicsprocessors for use within an SoC, according to embodiments describedherein, to support and/or to enable the multiple mode cooling subsystem.In an example, the graphic processors may be used in the intelligentcontrol of the multiple mode cooling subsystem because of existing mathengines capable of faster processing of multi-level neural networks.FIG. 10B illustrates an exemplary graphics processor 1010 of a system ona chip integrated circuit that may be fabricated using one or more IPcores, according to at least one embodiment. FIG. 10C illustrates anadditional exemplary graphics processor 1040 of a system on a chipintegrated circuit that may be fabricated using one or more IP cores,according to at least one embodiment. In at least one embodiment,graphics processor 1010 of FIG. 10A is a low power graphics processorcore. In at least one embodiment, graphics processor 1040 of FIG. 10C isa higher performance graphics processor core. In at least oneembodiment, each of graphics processors 1010, 1040 can be variants ofgraphics processor 1010 of FIG. 10A.

In at least one embodiment, graphics processor 1010 includes a vertexprocessor 1005 and one or more fragment processor(s) 1015A-1015N (e.g.,1015A, 1015B, 1015C, 1015D, through 1015N-1, and 1015N). In at least oneembodiment, graphics processor 1010 can execute different shaderprograms via separate logic, such that vertex processor 1005 isoptimized to execute operations for vertex shader programs, while one ormore fragment processor(s) 1015A-1015N execute fragment (e.g., pixel)shading operations for fragment or pixel shader programs. In at leastone embodiment, vertex processor 1005 performs a vertex processing stageof a 3D graphics pipeline and generates primitives and vertex data. Inat least one embodiment, fragment processor(s) 1015A-1015N use primitiveand vertex data generated by vertex processor 1005 to produce aframebuffer that is displayed on a display device. In at least oneembodiment, fragment processor(s) 1015A-1015N are optimized to executefragment shader programs as provided for in an OpenGL API, which may beused to perform similar operations as a pixel shader program as providedfor in a Direct 3D API.

In at least one embodiment, graphics processor 1010 additionallyincludes one or more memory management units (MMUs) 1020A-1020B,cache(s) 1025A-1025B, and circuit interconnect(s) 1030A-1030B. In atleast one embodiment, one or more MMU(s) 1020A-1020B provide for virtualto physical address mapping for graphics processor 1010, including forvertex processor 1005 and/or fragment processor(s) 1015A-1015N, whichmay reference vertex or image/texture data stored in memory, in additionto vertex or image/texture data stored in one or more cache(s)1025A-1025B. In at least one embodiment, one or more MMU(s) 1020A-1020Bmay be synchronized with other MMUs within system, including one or moreMMUs associated with one or more application processor(s) 1005, imageprocessors 1015, and/or video processors 1020 of FIG. 10A, such thateach processor 1005-1020 can participate in a shared or unified virtualmemory system. In at least one embodiment, one or more circuitinterconnect(s) 1030A-1030B enable graphics processor 1010 to interfacewith other IP cores within SoC, either via an internal bus of SoC or viaa direct connection.

In at least one embodiment, graphics processor 1040 includes one or moreMMU(s) 1020A-1020B, cache(s) 1025A-1025B, and circuit interconnect(s)1030A-1030B of graphics processor 1010 of FIG. 10A. In at least oneembodiment, graphics processor 1040 includes one or more shader core(s)1055A-1055N (e.g., 1055A, 1055B, 1055C, 1055D, 1055E, 1055F, through1055N-1, and 1055N), which provides for a unified shader corearchitecture in which a single core or type or core can execute alltypes of programmable shader code, including shader program code toimplement vertex shaders, fragment shaders, and/or compute shaders. Inat least one embodiment, a number of shader cores can vary. In at leastone embodiment, graphics processor 1040 includes an inter-core taskmanager 1045, which acts as a thread dispatcher to dispatch executionthreads to one or more shader cores 1055A-1055N and a tiling unit 1058to accelerate tiling operations for tile-based rendering, in whichrendering operations for a scene are subdivided in image space, forexample to exploit local spatial coherence within a scene or to optimizeuse of internal caches.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,inference and/or training logic 615 may be used in integrated circuit10A and/or 10B for inferencing or predicting operations based, at leastin part, on weight parameters calculated using neural network trainingoperations, neural network functions and/or architectures, or neuralnetwork use cases described herein.

FIGS. 10D-10E illustrate additional exemplary graphics processor logicaccording to embodiments described herein to support and/or to enablethe multiple mode cooling subsystem. FIG. 10D illustrates a graphicscore 1000D that may be included within graphics processor 1010 of FIG.10A, in at least one embodiment, and may be a unified shader core1055A-1055N as in FIG. 10C in at least one embodiment. FIG. 10Billustrates a highly-parallel general-purpose graphics processing unit1030 suitable for deployment on a multi-chip module in at least oneembodiment.

In at least one embodiment, graphics core 1000D can include multipleslices 1001A-1001N or partition for each core, and a graphics processorcan include multiple instances of graphics core 1000D. Slices1001A-1001N can include support logic including a local instructioncache 1004A-1004N, a thread scheduler 1006A-1006N, a thread dispatcher1008A-1008N, and a set of registers 1010A-1010N. In at least oneembodiment, slices 1001A-1001N can include a set of additional functionunits (AFUs 1012A-1012N), floating-point units (FPU 1014A-1014N),integer arithmetic logic units (ALUs 1016-1016N), address computationalunits (ACU 1013A-1013N), double-precision floating-point units (DPFPU1015A-1015N), and matrix processing units (MPU 1017A-1017N).

In at least one embodiment, FPUs 1014A-1014N can performsingle-precision (32-bit) and half-precision (16-bit) floating pointoperations, while DPFPUs 1015A-1015N perform double precision (64-bit)floating point operations. In at least one embodiment, ALUs 1016A-1016Ncan perform variable precision integer operations at 8-bit, 16-bit, and32-bit precision, and can be configured for mixed precision operations.In at least one embodiment, MPUs 1017A-1017N can also be configured formixed precision matrix operations, including half-precision floatingpoint and 8-bit integer operations. In at least one embodiment, MPUs1017A-1017N can perform a variety of matrix operations to acceleratemachine learning application frameworks, including enabling support foraccelerated general matrix to matrix multiplication (GEMM). In at leastone embodiment, AFUs 1012A-1012N can perform additional logic operationsnot supported by floating-point or integer units, includingtrigonometric operations (e.g., Sine, Cosine, etc.).

As discussed elsewhere in this disclosure, inference and/or traininglogic 615 (referenced at least in FIGS. 6B, 6C) may be used to performinferencing and/or training operations associated with one or moreembodiments. Details regarding inference and/or training logic 615 areprovided below in conjunction with FIGS. 6B and/or 6C. In at least oneembodiment, inference and/or training logic 615 may be used in graphicscore 1000D for inferencing or predicting operations based, at least inpart, on weight parameters calculated using neural network trainingoperations, neural network functions and/or architectures, or neuralnetwork use cases described herein.

FIG. 11A is a block diagram illustrating a computing system 1100Aaccording to at least one embodiment. In at least one embodiment,computing system 1100A includes a processing subsystem 1101 having oneor more processor(s) 1102 and a system memory 1104 communicating via aninterconnection path that may include a memory hub 1105. In at least oneembodiment, memory hub 1105 may be a separate component within a chipsetcomponent or may be integrated within one or more processor(s) 1102. Inat least one embodiment, memory hub 1105 couples with an I/O subsystem1111 via a communication link 1106. In at least one embodiment, I/Osubsystem 1111 includes an I/O hub 1107 that can enable computing system1100A to receive input from one or more input device(s) 1108. In atleast one embodiment, I/O hub 1107 can enable a display controller,which may be included in one or more processor(s) 1102, to provideoutputs to one or more display device(s) 1110A. In at least oneembodiment, one or more display device(s) 1110A coupled with I/O hub1107 can include a local, internal, or embedded display device.

In at least one embodiment, processing subsystem 1101 includes one ormore parallel processor(s) 1112 coupled to memory hub 1105 via a bus orother communication link 1113. In at least one embodiment, communicationlink 1113 may be one of any number of standards based communication linktechnologies or protocols, such as but not limited to PCI Express, ormay be a vendor specific communications interface or communicationsfabric. In at least one embodiment, one or more parallel processor(s)1112 form a computationally focused parallel or vector processing systemthat can include a large number of processing cores and/or processingclusters, such as a many integrated core (MIC) processor. In at leastone embodiment, one or more parallel processor(s) 1112 form a graphicsprocessing subsystem that can output pixels to one of one or moredisplay device(s) 1110A coupled via I/O Hub 1107. In at least oneembodiment, one or more parallel processor(s) 1112 can also include adisplay controller and display interface (not shown) to enable a directconnection to one or more display device(s) 1110B.

In at least one embodiment, a system storage unit 1114 can connect toI/O hub 1107 to provide a storage mechanism for computing system 1100A.In at least one embodiment, an I/O switch 1116 can be used to provide aninterface mechanism to enable connections between I/O hub 1107 and othercomponents, such as a network adapter 1118 and/or wireless networkadapter 1119 that may be integrated into a platform(s), and variousother devices that can be added via one or more add-in device(s) 1120.In at least one embodiment, network adapter 1118 can be an Ethernetadapter or another wired network adapter. In at least one embodiment,wireless network adapter 1119 can include one or more of a Wi-Fi,Bluetooth, near field communication (NFC), or other network device thatincludes one or more wireless radios.

In at least one embodiment, computing system 1100A can include othercomponents not explicitly shown, including USB or other portconnections, optical storage drives, video capture devices, and so on,may also be connected to I/O hub 1107. In at least one embodiment,communication paths interconnecting various components in FIG. 11A maybe implemented using any suitable protocols, such as PCI (PeripheralComponent Interconnect) based protocols (e.g., PCI-Express), or otherbus or point-to-point communication interfaces and/or protocol(s), suchas NV-Link high-speed interconnect, or interconnect protocols.

In at least one embodiment, one or more parallel processor(s) 1112incorporate circuitry optimized for graphics and video processing,including, for example, video output circuitry, and constitutes agraphics processing unit (GPU). In at least one embodiment, one or moreparallel processor(s) 1112 incorporate circuitry optimized for generalpurpose processing. In at least one embodiment, components of computingsystem 1100A may be integrated with one or more other system elements ona single integrated circuit. In at least one embodiment, in at least oneembodiment, one or more parallel processor(s) 1112, memory hub 1105,processor(s) 1102, and I/O hub 1107 can be integrated into a system onchip (SoC) integrated circuit. In at least one embodiment, components ofcomputing system 1100A can be integrated into a single package to form asystem in package (SIP) configuration. In at least one embodiment, atleast a portion of components of computing system 1100A can beintegrated into a multi-chip module (MCM), which can be interconnectedwith other multi-chip modules into a modular computing system.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,inference and/or training logic 615 may be used in system FIG. 11A forinferencing or predicting operations based, at least in part, on weightparameters calculated using neural network training operations, neuralnetwork functions and/or architectures, or neural network use casesdescribed herein.

Processors

FIG. 11B illustrates a parallel processor 1100B according to at leastone embodiment. In at least one embodiment, various components ofparallel processor 1100B may be implemented using one or more integratedcircuit devices, such as programmable processors, application specificintegrated circuits (ASICs), or field programmable gate arrays (FPGA).In at least one embodiment, illustrated parallel processor 11100B 100 isa variant of one or more parallel processor(s) 1112 shown in FIG. 11Baccording to an exemplary embodiment.

In at least one embodiment, parallel processor 1100B includes a parallelprocessing unit 1102. In at least one embodiment, parallel processingunit 1102 includes an I/O unit 1104 that enables communication withother devices, including other instances of parallel processing unit1102. In at least one embodiment, I/O unit 1104 may be directlyconnected to other devices. In at least one embodiment, I/O unit 1104connects with other devices via use of a hub or switch interface, suchas memory hub 1105. In at least one embodiment, connections betweenmemory hub 1105 and I/O unit 1104 form a communication link 1113. In atleast one embodiment, I/O unit 1104 connects with a host interface 1106and a memory crossbar 1116, where host interface 1106 receives commandsdirected to performing processing operations and memory crossbar 1116receives commands directed to performing memory operations.

In at least one embodiment, when host interface 1106 receives a commandbuffer via I/O unit 1104, host interface 1106 can direct work operationsto perform those commands to a front end 1108. In at least oneembodiment, front end 1108 couples with a scheduler 1110, which isconfigured to distribute commands or other work items to a processingcluster array 1112. In at least one embodiment, scheduler 1110 ensuresthat processing cluster array 1112 is properly configured and in a validstate before tasks are distributed to processing cluster array 1112. Inat least one embodiment, scheduler 1110 is implemented via firmwarelogic executing on a microcontroller. In at least one embodiment,microcontroller implemented scheduler 1110 is configurable to performcomplex scheduling and work distribution operations at coarse and finegranularity, enabling rapid preemption and context switching of threadsexecuting on processing array 1112. In at least one embodiment, hostsoftware can prove workloads for scheduling on processing array 1112 viaone of multiple graphics processing doorbells. In at least oneembodiment, workloads can then be automatically distributed acrossprocessing array 1112 by scheduler 1110 logic within a microcontrollerincluding scheduler 1110.

In at least one embodiment, processing cluster array 1112 can include upto “N” processing clusters (e.g., cluster 1114A, cluster 1114B, throughcluster 1114N). In at least one embodiment, each cluster 1114A-1114N ofprocessing cluster array 1112 can execute a large number of concurrentthreads. In at least one embodiment, scheduler 1110 can allocate work toclusters 1114A-1114N of processing cluster array 1112 using variousscheduling and/or work distribution algorithms, which may vary dependingon workload arising for each type of program or computation. In at leastone embodiment, scheduling can be handled dynamically by scheduler 1110,or can be assisted in part by compiler logic during compilation ofprogram logic configured for execution by processing cluster array 1112.In at least one embodiment, different clusters 1114A-1114N of processingcluster array 1112 can be allocated for processing different types ofprograms or for performing different types of computations.

In at least one embodiment, processing cluster array 1112 can beconfigured to perform various types of parallel processing operations.In at least one embodiment, processing cluster array 1112 is configuredto perform general-purpose parallel compute operations. In at least oneembodiment, in at least one embodiment, processing cluster array 1112can include logic to execute processing tasks including filtering ofvideo and/or audio data, performing modeling operations, includingphysics operations, and performing data transformations.

In at least one embodiment, processing cluster array 1112 is configuredto perform parallel graphics processing operations. In at least oneembodiment, processing cluster array 1112 can include additional logicto support execution of such graphics processing operations, including,but not limited to texture sampling logic to perform texture operations,as well as tessellation logic and other vertex processing logic. In atleast one embodiment, processing cluster array 1112 can be configured toexecute graphics processing related shader programs such as but notlimited to vertex shaders, tessellation shaders, geometry shaders, andpixel shaders. In at least one embodiment, parallel processing unit 1102can transfer data from system memory via I/O unit 1104 for processing.In at least one embodiment, during processing, transferred data can bestored to on-chip memory (e.g., parallel processor memory 1122) duringprocessing, then written back to system memory.

In at least one embodiment, when parallel processing unit 1102 is usedto perform graphics processing, scheduler 1110 can be configured todivide a processing workload into approximately equal sized tasks, tobetter enable distribution of graphics processing operations to multipleclusters 1114A-1114N of processing cluster array 1112. In at least oneembodiment, portions of processing cluster array 1112 can be configuredto perform different types of processing. In at least one embodiment, inat least one embodiment, a first portion may be configured to performvertex shading and topology generation, a second portion may beconfigured to perform tessellation and geometry shading, and a thirdportion may be configured to perform pixel shading or other screen spaceoperations, to produce a rendered image for display if a simulation ofvalve control for the multiple mode cooling subsystem is required. In atleast one embodiment, intermediate data produced by one or more ofclusters 1114A-1114N may be stored in buffers to allow intermediate datato be transmitted between clusters 1114A-1114N for further processing.

In at least one embodiment, processing cluster array 1112 can receiveprocessing tasks to be executed via scheduler 1110, which receivescommands defining processing tasks from front end 1108. In at least oneembodiment, processing tasks can include indices of data to beprocessed, e.g., surface (patch) data, primitive data, vertex data,and/or pixel data, as well as state parameters and commands defining howdata is to be processed (e.g., what program is to be executed). In atleast one embodiment, scheduler 1110 may be configured to fetch indicescorresponding to tasks or may receive indices from front end 1108. In atleast one embodiment, front end 1108 can be configured to ensureprocessing cluster array 1112 is configured to a valid state before aworkload specified by incoming command buffers (e.g., batch-buffers,push buffers, etc.) is initiated.

In at least one embodiment, each of one or more instances of parallelprocessing unit 1102 can couple with parallel processor memory 1122. Inat least one embodiment, parallel processor memory 1122 can be accessedvia memory crossbar 1116, which can receive memory requests fromprocessing cluster array 1112 as well as I/O unit 1104. In at least oneembodiment, memory crossbar 1116 can access parallel processor memory1122 via a memory interface 1118. In at least one embodiment, memoryinterface 1118 can include multiple partition units (e.g., partitionunit 1120A, partition unit 1120B, through partition unit 1120N) that caneach couple to a portion (e.g., memory unit) of parallel processormemory 1122. In at least one embodiment, a number of partition units1120A-1120N is configured to be equal to a number of memory units, suchthat a first partition unit 1120A has a corresponding first memory unit1124A, a second partition unit 1120B has a corresponding memory unit1124B, and a Nth partition unit 1120N has a corresponding Nth memoryunit 1124N. In at least one embodiment, a number of partition units1120A-1120N may not be equal to a number of memory devices.

In at least one embodiment, memory units 1124A-1124N can include varioustypes of memory devices, including dynamic random access memory (DRAM)or graphics random access memory, such as synchronous graphics randomaccess memory (SGRAM), including graphics double data rate (GDDR)memory. In at least one embodiment, memory units 1124A-1124N may alsoinclude 3D stacked memory, including but not limited to high bandwidthmemory (HBM). In at least one embodiment, render targets, such as framebuffers or texture maps may be stored across memory units 1124A-1124N,allowing partition units 1120A-1120N to write portions of each rendertarget in parallel to efficiently use available bandwidth of parallelprocessor memory 1122. In at least one embodiment, a local instance ofparallel processor memory 1122 may be excluded in favor of a unifiedmemory design that utilizes system memory in conjunction with localcache memory.

In at least one embodiment, any one of clusters 1114A-1114N ofprocessing cluster array 1112 can process data that will be written toany of memory units 1124A-1124N within parallel processor memory 1122.In at least one embodiment, memory crossbar 1116 can be configured totransfer an output of each cluster 1114A-1114N to any partition unit1120A-1120N or to another cluster 1114A-1114N, which can performadditional processing operations on an output. In at least oneembodiment, each cluster 1114A-1114N can communicate with memoryinterface 1118 through memory crossbar 1116 to read from or write tovarious external memory devices. In at least one embodiment, memorycrossbar 1116 has a connection to memory interface 1118 to communicatewith I/O unit 1104, as well as a connection to a local instance ofparallel processor memory 1122, enabling processing units withindifferent processing clusters 1114A-1114N to communicate with systemmemory or other memory that is not local to parallel processing unit1102. In at least one embodiment, memory crossbar 1116 can use virtualchannels to separate traffic streams between clusters 1114A-1114N andpartition units 1120A-1120N.

In at least one embodiment, multiple instances of parallel processingunit 1102 can be provided on a single add-in card, or multiple add-incards can be interconnected. In at least one embodiment, differentinstances of parallel processing unit 1102 can be configured tointer-operate even if different instances have different numbers ofprocessing cores, different amounts of local parallel processor memory,and/or other configuration differences. In at least one embodiment, inat least one embodiment, some instances of parallel processing unit 1102can include higher precision floating point units relative to otherinstances. In at least one embodiment, systems incorporating one or moreinstances of parallel processing unit 1102 or parallel processor 1100Bcan be implemented in a variety of configurations and form factors,including but not limited to desktop, laptop, or handheld personalcomputers, servers, workstations, game consoles, and/or embeddedsystems.

FIG. 11C is a block diagram of a partition unit 1120 according to atleast one embodiment. In at least one embodiment, partition unit 1120 isan instance of one of partition units 1120A-1120N of FIG. 11B. In atleast one embodiment, partition unit 1120 includes an L2 cache 1121, aframe buffer interface 1125, and a raster operations unit (“ROP”) 1126.L2 cache 1121 is a read/write cache that is configured to perform loadand store operations received from memory crossbar 1116 and ROP 1126. Inat least one embodiment, read misses and urgent write-back requests areoutput by L2 cache 1121 to frame buffer interface 1125 for processing.In at least one embodiment, updates can also be sent to a frame buffervia frame buffer interface 1125 for processing. In at least oneembodiment, frame buffer interface 1125 interfaces with one of memoryunits in parallel processor memory, such as memory units 1124A-1124N ofFIG. 11B (e.g., within parallel processor memory 1122).

In at least one embodiment, ROP 1126 is a processing unit that performsraster operations such as stencil, z test, blending, and so forth. In atleast one embodiment, ROP 1126 then outputs processed graphics data thatis stored in graphics memory. In at least one embodiment, ROP 1126includes compression logic to compress depth or color data that iswritten to memory and decompress depth or color data that is read frommemory. In at least one embodiment, compression logic can be losslesscompression logic that makes use of one or more of multiple compressionalgorithms. Compression logic that is performed by ROP 1126 can varybased on statistical characteristics of data to be compressed. In atleast one embodiment, in at least one embodiment, delta colorcompression is performed on depth and color data on a per-tile basis.

In at least one embodiment, ROP 1126 is included within each processingcluster (e.g., cluster 1114A-1114N of FIG. 11B) instead of withinpartition unit 1120. In at least one embodiment, read and write requestsfor pixel data are transmitted over memory crossbar 1116 instead ofpixel fragment data. In at least one embodiment, processed graphics datamay be displayed on a display device, such as one of one or more displaydevice(s) 1110 of FIG. 11, routed for further processing by processor(s)1102, or routed for further processing by one of processing entitieswithin parallel processor 1100B of FIG. 11B.

FIG. 11D is a block diagram of a processing cluster 1114 within aparallel processing unit according to at least one embodiment. In atleast one embodiment, a processing cluster is an instance of one ofprocessing clusters 1114A-1114N of FIG. 11B. In at least one embodiment,one of more of processing cluster(s) 1114 can be configured to executemany threads in parallel, where “thread” refers to an instance of aparticular program executing on a particular set of input data. In atleast one embodiment, single-instruction, multiple-data (SIMD)instruction issue techniques are used to support parallel execution of alarge number of threads without providing multiple independentinstruction units. In at least one embodiment, single-instruction,multiple-thread (SIMT) techniques are used to support parallel executionof a large number of synchronized threads, using a common instructionunit configured to issue instructions to a set of processing engineswithin each one of processing clusters.

In at least one embodiment, operation of processing cluster 1114 can becontrolled via a pipeline manager 1132 that distributes processing tasksto SIMT parallel processors. In at least one embodiment, pipelinemanager 1132 receives instructions from scheduler 1110 of FIG. 11B andmanages execution of those instructions via a graphics multiprocessor1134 and/or a texture unit 1136. In at least one embodiment, graphicsmultiprocessor 1134 is an exemplary instance of a SIMT parallelprocessor. However, in at least one embodiment, various types of SIMTparallel processors of differing architectures may be included withinprocessing cluster 1114. In at least one embodiment, one or moreinstances of graphics multiprocessor 1134 can be included within aprocessing cluster 1114. In at least one embodiment, graphicsmultiprocessor 1134 can process data and a data crossbar 1140 can beused to distribute processed data to one of multiple possibledestinations, including other shader units. In at least one embodiment,pipeline manager 1132 can facilitate distribution of processed data byspecifying destinations for processed data to be distributed vis datacrossbar 1140.

In at least one embodiment, each graphics multiprocessor 1134 withinprocessing cluster 1114 can include an identical set of functionalexecution logic (e.g., arithmetic logic units, load-store units, etc.).In at least one embodiment, functional execution logic can be configuredin a pipelined manner in which new instructions can be issued beforeprevious instructions are complete. In at least one embodiment,functional execution logic supports a variety of operations includinginteger and floating point arithmetic, comparison operations, Booleanoperations, bit-shifting, and computation of various algebraicfunctions. In at least one embodiment, same functional-unit hardware canbe leveraged to perform different operations and any combination offunctional units may be present.

In at least one embodiment, instructions transmitted to processingcluster 1114 constitute a thread. In at least one embodiment, a set ofthreads executing across a set of parallel processing engines is athread group. In at least one embodiment, thread group executes aprogram on different input data. In at least one embodiment, each threadwithin a thread group can be assigned to a different processing enginewithin a graphics multiprocessor 1134. In at least one embodiment, athread group may include fewer threads than a number of processingengines within graphics multiprocessor 1134. In at least one embodiment,when a thread group includes fewer threads than a number of processingengines, one or more processing engines may be idle during cycles inwhich that thread group is being processed. In at least one embodiment,a thread group may also include more threads than a number of processingengines within graphics multiprocessor 1134. In at least one embodiment,when a thread group includes more threads than processing engines withingraphics multiprocessor 1134, processing can be performed overconsecutive clock cycles. In at least one embodiment, multiple threadgroups can be executed concurrently on a graphics multiprocessor 1134.

In at least one embodiment, graphics multiprocessor 1134 includes aninternal cache memory to perform load and store operations. In at leastone embodiment, graphics multiprocessor 1134 can forego an internalcache and use a cache memory (e.g., L1 cache 1148) within processingcluster 1114. In at least one embodiment, each graphics multiprocessor1134 also has access to L2 caches within partition units (e.g.,partition units 1120A-1120N of FIG. 11B) that are shared among allprocessing clusters 1114 and may be used to transfer data betweenthreads. In at least one embodiment, graphics multiprocessor 1134 mayalso access off-chip global memory, which can include one or more oflocal parallel processor memory and/or system memory. In at least oneembodiment, any memory external to parallel processing unit 1102 may beused as global memory. In at least one embodiment, processing cluster1114 includes multiple instances of graphics multiprocessor 1134 canshare common instructions and data, which may be stored in L1 cache1148.

In at least one embodiment, each processing cluster 1114 may include amemory management unit (“MMU”) 1145 that is configured to map virtualaddresses into physical addresses. In at least one embodiment, one ormore instances of MMU 1145 may reside within memory interface 1118 ofFIG. 11B. In at least one embodiment, MMU 1145 includes a set of pagetable entries (PTEs) used to map a virtual address to a physical addressof a tile and, in at least one embodiment, a cache line index. In atleast one embodiment, MMU 1145 may include address translation lookasidebuffers (TLB) or caches that may reside within graphics multiprocessor1134 or L1 cache or processing cluster 1114. In at least one embodiment,physical address is processed to distribute surface data access localityto allow efficient request interleaving among partition units. In atleast one embodiment, cache line index may be used to determine whethera request for a cache line is a hit or miss.

In at least one embodiment, a processing cluster 1114 may be configuredsuch that each graphics multiprocessor 1134 is coupled to a texture unit1136 for performing texture mapping operations, e.g., determiningtexture sample positions, reading texture data, and filtering texturedata. In at least one embodiment, texture data is read from an internaltexture L1 cache (not shown) or from an L1 cache within graphicsmultiprocessor 1134 and is fetched from an L2 cache, local parallelprocessor memory, or system memory, as needed. In at least oneembodiment, each graphics multiprocessor 1134 outputs processed tasks todata crossbar 1140 to provide processed task(s) to another processingcluster 1114 for further processing or to store processed task(s) in anL2 cache, local parallel processor memory, or system memory via memorycrossbar 1116. In at least one embodiment, preROP 1142 (pre-rasteroperations unit) is configured to receive data from graphicsmultiprocessor 1134, direct data to ROP units, which may be located withpartition units as described herein (e.g., partition units 1120A-1120Nof FIG. 11B). In at least one embodiment, PreROP 1142 unit can performoptimizations for color blending, organize pixel color data, and performaddress translations.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,inference and/or training logic 615 may be used in graphics processingcluster 1114 for inferencing or predicting operations based, at least inpart, on weight parameters calculated using neural network trainingoperations, neural network functions and/or architectures, or neuralnetwork use cases described herein.

FIG. 11E shows a graphics multiprocessor 1134 according to at least oneembodiment. In at least one embodiment, graphics multiprocessor 1134couples with pipeline manager 1132 of processing cluster 1114. In atleast one embodiment, graphics multiprocessor 1134 has an executionpipeline including but not limited to an instruction cache 1152, aninstruction unit 1154, an address mapping unit 1156, a register file1158, one or more general purpose graphics processing unit (GPGPU) cores1162, and one or more load/store units 1166. GPGPU core(s) 1162 andload/store unit(s) 1166 are coupled with cache memory 1172 and sharedmemory 1170 via a memory and cache interconnect 1168.

In at least one embodiment, instruction cache 1152 receives a stream ofinstructions to execute from pipeline manager 1132. In at least oneembodiment, instructions are cached in instruction cache 1152 anddispatched for execution by instruction unit 1154. In at least oneembodiment, instruction unit 1154 can dispatch instructions as threadgroups (e.g., warps), with each thread group assigned to a differentexecution unit within GPGPU core(s) 1162. In at least one embodiment, aninstruction can access any of a local, shared, or global address spaceby specifying an address within a unified address space. In at least oneembodiment, address mapping unit 1156 can be used to translate addressesin a unified address space into a distinct memory address that can beaccessed by load/store unit(s) 1166.

In at least one embodiment, register file 1158 provides a set ofregisters for functional units of graphics multiprocessor 1134. In atleast one embodiment, register file 1158 provides temporary storage foroperands connected to data paths of functional units (e.g., GPGPU cores1162, load/store units 1166) of graphics multiprocessor 1134. In atleast one embodiment, register file 1158 is divided between each offunctional units such that each functional unit is allocated a dedicatedportion of register file 1158. In at least one embodiment, register file1158 is divided between different warps being executed by graphicsmultiprocessor 1134.

In at least one embodiment, GPGPU cores 1162 can each include floatingpoint units (FPUs) and/or integer arithmetic logic units (ALUs) that areused to execute instructions of graphics multiprocessor 1134. GPGPUcores 1162 can be similar in architecture or can differ in architecture.In at least one embodiment, a first portion of GPGPU cores 1162 includea single precision FPU and an integer ALU while a second portion ofGPGPU cores include a double precision FPU. In at least one embodiment,FPUs can implement IEEE 754-2008 standard for floating point arithmeticor enable variable precision floating point arithmetic. In at least oneembodiment, graphics multiprocessor 1134 can additionally include one ormore fixed function or special function units to perform specificfunctions such as copy rectangle or pixel blending operations. In atleast one embodiment one or more of GPGPU cores can also include fixedor special function logic.

In at least one embodiment, GPGPU cores 1162 include SIMD logic capableof performing a single instruction on multiple sets of data. In at leastone embodiment GPGPU cores 1162 can physically execute SIMD4, SIMD8, andSIMD16 instructions and logically execute SIMD1, SIMD2, and SIMD32instructions. In at least one embodiment, SIMD instructions for GPGPUcores can be generated at compile time by a shader compiler orautomatically generated when executing programs written and compiled forsingle program multiple data (SPMD) or SIMT architectures. In at leastone embodiment, multiple threads of a program configured for an SIMTexecution model can executed via a single SIMD instruction. In at leastone embodiment, in at least one embodiment, eight SIMT threads thatperform same or similar operations can be executed in parallel via asingle SIMD8 logic unit.

In at least one embodiment, memory and cache interconnect 1168 is aninterconnect network that connects each functional unit of graphicsmultiprocessor 1134 to register file 1158 and to shared memory 1170. Inat least one embodiment, memory and cache interconnect 1168 is acrossbar interconnect that allows load/store unit 1166 to implement loadand store operations between shared memory 1170 and register file 1158.In at least one embodiment, register file 1158 can operate at a samefrequency as GPGPU cores 1162, thus data transfer between GPGPU cores1162 and register file 1158 is very low latency. In at least oneembodiment, shared memory 1170 can be used to enable communicationbetween threads that execute on functional units within graphicsmultiprocessor 1134. In at least one embodiment, cache memory 1172 canbe used as a data cache for example, to cache texture data communicatedbetween functional units and texture unit 1136. In at least oneembodiment, shared memory 1170 can also be used as a program managedcache. In at least one embodiment, threads executing on GPGPU cores 1162can programmatically store data within shared memory in addition toautomatically cached data that is stored within cache memory 1172.

In at least one embodiment, a parallel processor or GPGPU as describedherein is communicatively coupled to host/processor cores to accelerategraphics operations, machine-learning operations, pattern analysisoperations, and various general purpose GPU (GPGPU) functions. In atleast one embodiment, GPU may be communicatively coupled to hostprocessor/cores over a bus or other interconnect (e.g., a high speedinterconnect such as PCIe or NVLink). In at least one embodiment, GPUmay be integrated on same package or chip as cores and communicativelycoupled to cores over an internal processor bus/interconnect (in atleast one embodiment, internal to package or chip). In at least oneembodiment, regardless of manner in which GPU is connected, processorcores may allocate work to GPU in form of sequences ofcommands/instructions contained in a work descriptor. In at least oneembodiment, GPU then uses dedicated circuitry/logic for efficientlyprocessing these commands/instructions.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,inference and/or training logic 615 may be used in graphicsmultiprocessor 1134 for inferencing or predicting operations based, atleast in part, on weight parameters calculated using neural networktraining operations, neural network functions and/or architectures, orneural network use cases described herein.

FIG. 12A illustrates a multi-GPU computing system 1200A, according to atleast one embodiment. In at least one embodiment, multi-GPU computingsystem 1200A can include a processor 1202 coupled to multiple generalpurpose graphics processing units (GPGPUs) 1206A-D via a host interfaceswitch 1204. In at least one embodiment, host interface switch 1204 is aPCI express switch device that couples processor 1202 to a PCI expressbus over which processor 1202 can communicate with GPGPUs 1206A-D.GPGPUs 1206A-D can interconnect via a set of high-speed point to pointGPU to GPU links 1216. In at least one embodiment, GPU to GPU links 1216connect to each of GPGPUs 1206A-D via a dedicated GPU link. In at leastone embodiment, P2P GPU links 1216 enable direct communication betweeneach of GPGPUs 1206A-D without requiring communication over hostinterface bus 1204 to which processor 1202 is connected. In at least oneembodiment, with GPU-to-GPU traffic directed to P2P GPU links 1216, hostinterface bus 1204 remains available for system memory access or tocommunicate with other instances of multi-GPU computing system 1200A,for example, via one or more network devices. While in at least oneembodiment GPGPUs 1206A-D connect to processor 1202 via host interfaceswitch 1204, in at least one embodiment processor 1202 includes directsupport for P2P GPU links 1216 and can connect directly to GPGPUs1206A-D.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,inference and/or training logic 615 may be used in multi-GPU computingsystem 1200A for inferencing or predicting operations based, at least inpart, on weight parameters calculated using neural network trainingoperations, neural network functions and/or architectures, or neuralnetwork use cases described herein.

FIG. 12B is a block diagram of a graphics processor 1200B, according toat least one embodiment. In at least one embodiment, graphics processor1200B includes a ring interconnect 1202, a pipeline front-end 1204, amedia engine 1237, and graphics cores 1280A-1280N. In at least oneembodiment, ring interconnect 1202 couples graphics processor 1200B toother processing units, including other graphics processors or one ormore general-purpose processor cores. In at least one embodiment,graphics processor 1200B is one of many processors integrated within amulti-core processing system.

In at least one embodiment, graphics processor 1200B receives batches ofcommands via ring interconnect 1202. In at least one embodiment,incoming commands are interpreted by a command streamer 1203 in pipelinefront-end 1204. In at least one embodiment, graphics processor 1200Bincludes scalable execution logic to perform 3D geometry processing andmedia processing via graphics core(s) 1280A-1280N. In at least oneembodiment, for 3D geometry processing commands, command streamer 1203supplies commands to geometry pipeline 1236. In at least one embodiment,for at least some media processing commands, command streamer 1203supplies commands to a video front end 1234, which couples with a mediaengine 1237. In at least one embodiment, media engine 1237 includes aVideo Quality Engine (VQE) 1230 for video and image post-processing anda multi-format encode/decode (MFX) 1233 engine to providehardware-accelerated media data encode and decode. In at least oneembodiment, geometry pipeline 1236 and media engine 1237 each generateexecution threads for thread execution resources provided by at leastone graphics core 1280A.

In at least one embodiment, graphics processor 1200B includes scalablethread execution resources featuring modular cores 1280A-1280N(sometimes referred to as core slices), each having multiple sub-cores1250A-1250N, 1260A-1260N (sometimes referred to as core sub-slices). Inat least one embodiment, graphics processor 1200B can have any number ofgraphics cores 1280A through 1280N. In at least one embodiment, graphicsprocessor 1200B includes a graphics core 1280A having at least a firstsub-core 1250A and a second sub-core 1260A. In at least one embodiment,graphics processor 1200B is a low power processor with a single sub-core(e.g., 1250A). In at least one embodiment, graphics processor 1200Bincludes multiple graphics cores 1280A-1280N, each including a set offirst sub-cores 1250A-1250N and a set of second sub-cores 1260A-1260N.In at least one embodiment, each sub-core in first sub-cores 1250A-1250Nincludes at least a first set of execution units 1252A-1252N andmedia/texture samplers 1254A-1254N. In at least one embodiment, eachsub-core in second sub-cores 1260A-1260N includes at least a second setof execution units 1262A-1262N and samplers 1264A-1264N. In at least oneembodiment, each sub-core 1250A-1250N, 1260A-1260N shares a set ofshared resources 1270A-1270N. In at least one embodiment, sharedresources include shared cache memory and pixel operation logic.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,inference and/or training logic 615 may be used in graphics processor1200B for inferencing or predicting operations based, at least in part,on weight parameters calculated using neural network trainingoperations, neural network functions and/or architectures, or neuralnetwork use cases described herein.

FIG. 13 is a block diagram illustrating micro-architecture for aprocessor 1300 that may include logic circuits to perform instructions,according to at least one embodiment. In at least one embodiment,processor 1300 may perform instructions, including x86 instructions, ARMinstructions, specialized instructions for application-specificintegrated circuits (ASICs), etc. In at least one embodiment, processor1300 may include registers to store packed data, such as 64-bit wideMMX™ registers in microprocessors enabled with MMX technology from IntelCorporation of Santa Clara, Calif. In at least one embodiment, MMXregisters, available in both integer and floating point forms, mayoperate with packed data elements that accompany single instruction,multiple data (“SIMD”) and streaming SIMD extensions (“SSE”)instructions. In at least one embodiment, 128-bit wide XMM registersrelating to SSE2, SSE3, SSE4, AVX, or beyond (referred to generically as“SSEx”) technology may hold such packed data operands. In at least oneembodiment, processor 1300 may perform instructions to acceleratemachine learning or deep learning algorithms, training, or inferencing.

In at least one embodiment, processor 1300 includes an in-order frontend (“front end”) 1301 to fetch instructions to be executed and prepareinstructions to be used later in processor pipeline. In at least oneembodiment, front end 1301 may include several units. In at least oneembodiment, an instruction prefetcher 1326 fetches instructions frommemory and feeds instructions to an instruction decoder 1328 which inturn decodes or interprets instructions. In at least one embodiment, inat least one embodiment, instruction decoder 1328 decodes a receivedinstruction into one or more operations called “micro-instructions” or“micro-operations” (also called “micro ops” or “uops”) that machine mayexecute. In at least one embodiment, instruction decoder 1328 parsesinstruction into an opcode and corresponding data and control fieldsthat may be used by micro-architecture to perform operations inaccordance with at least one embodiment. In at least one embodiment, atrace cache 1330 may assemble decoded uops into program orderedsequences or traces in a uop queue 1334 for execution. In at least oneembodiment, when trace cache 1330 encounters a complex instruction, amicrocode ROM 1332 provides uops needed to complete operation.

In at least one embodiment, some instructions may be converted into asingle micro-op, whereas others need several micro-ops to complete fulloperation. In at least one embodiment, if more than four micro-ops areneeded to complete an instruction, instruction decoder 1328 may accessmicrocode ROM 1332 to perform instruction. In at least one embodiment,an instruction may be decoded into a small number of micro-ops forprocessing at instruction decoder 1328. In at least one embodiment, aninstruction may be stored within microcode ROM 1332 should a number ofmicro-ops be needed to accomplish operation. In at least one embodiment,trace cache 1330 refers to an entry point programmable logic array(“PLA”) to determine a correct micro-instruction pointer for readingmicrocode sequences to complete one or more instructions from microcodeROM 1332 in accordance with at least one embodiment. In at least oneembodiment, after microcode ROM 1332 finishes sequencing micro-ops foran instruction, front end 1301 of machine may resume fetching micro-opsfrom trace cache 1330.

In at least one embodiment, out-of-order execution engine (“out of orderengine”) 1303 may prepare instructions for execution. In at least oneembodiment, out-of-order execution logic has a number of buffers tosmooth out and re-order flow of instructions to optimize performance asthey go down pipeline and get scheduled for execution. In at least oneembodiment, out-of-order execution engine 1303 includes, withoutlimitation, an allocator/register renamer 1340, a memory uop queue 1342,an integer/floating point uop queue 1344, a memory scheduler 1346, afast scheduler 1302, a slow/general floating point scheduler(“slow/general FP scheduler”) 1304, and a simple floating pointscheduler (“simple FP scheduler”) 1306. In at least one embodiment, fastschedule 1302, slow/general floating point scheduler 1304, and simplefloating point scheduler 1306 are also collectively referred to hereinas “uop schedulers 1302, 1304, 1306.” In at least one embodiment,allocator/register renamer 1340 allocates machine buffers and resourcesthat each uop needs in order to execute. In at least one embodiment,allocator/register renamer 1340 renames logic registers onto entries ina register file. In at least one embodiment, allocator/register renamer1340 also allocates an entry for each uop in one of two uop queues,memory uop queue 1342 for memory operations and integer/floating pointuop queue 1344 for non-memory operations, in front of memory scheduler1346 and uop schedulers 1302, 1304, 1306. In at least one embodiment,uop schedulers 1302, 1304, 1306 determine when a uop is ready to executebased on readiness of their dependent input register operand sources andavailability of execution resources uops need to complete theiroperation. In at least one embodiment, fast scheduler 1302 of at leastone embodiment may schedule on each half of main clock cycle whileslow/general floating point scheduler 1304 and simple floating pointscheduler 1306 may schedule once per main processor clock cycle. In atleast one embodiment, uop schedulers 1302, 1304, 1306 arbitrate fordispatch ports to schedule uops for execution.

In at least one embodiment, execution block 1311 includes, withoutlimitation, an integer register file/bypass network 1308, a floatingpoint register file/bypass network (“FP register file/bypass network”)1310, address generation units (“AGUs”) 1312 and 1314, fast ArithmeticLogic Units (ALUs) (“fast ALUs”) 1316 and 1318, a slow Arithmetic LogicUnit (“slow ALU”) 1320, a floating point ALU (“FP”) 1322, and a floatingpoint move unit (“FP move”) 1324. In at least one embodiment, integerregister file/bypass network 1308 and floating point registerfile/bypass network 1310 are also referred to herein as “register files1308, 1310.” In at least one embodiment, AGUs 1312 and 1314, fast ALUs1316 and 1318, slow ALU 1320, floating point ALU 1322, and floatingpoint move unit 1324 are also referred to herein as “execution units1312, 1314, 1316, 1318, 1320, 1322, and 1324.” In at least oneembodiment, execution block b11 may include, without limitation, anynumber (including zero) and type of register files, bypass networks,address generation units, and execution units, in any combination.

In at least one embodiment, register files 1308, 1310 may be arrangedbetween uop schedulers 1302, 1304, 1306, and execution units 1312, 1314,1316, 1318, 1320, 1322, and 1324. In at least one embodiment, integerregister file/bypass network 1308 performs integer operations. In atleast one embodiment, floating point register file/bypass network 1310performs floating point operations. In at least one embodiment, each ofregister files 1308, 1310 may include, without limitation, a bypassnetwork that may bypass or forward just completed results that have notyet been written into register file to new dependent uops. In at leastone embodiment, register files 1308, 1310 may communicate data with eachother. In at least one embodiment, integer register file/bypass network1308 may include, without limitation, two separate register files, oneregister file for low-order thirty-two bits of data and a secondregister file for high order thirty-two bits of data. In at least oneembodiment, floating point register file/bypass network 1310 mayinclude, without limitation, 128-bit wide entries because floating pointinstructions typically have operands from 64 to 128 bits in width.

In at least one embodiment, execution units 1312, 1314, 1316, 1318,1320, 1322, 1324 may execute instructions. In at least one embodiment,register files 1308, 1310 store integer and floating point data operandvalues that micro-instructions need to execute. In at least oneembodiment, processor 1300 may include, without limitation, any numberand combination of execution units 1312, 1314, 1316, 1318, 1320, 1322,1324. In at least one embodiment, floating point ALU 1322 and floatingpoint move unit 1324, may execute floating point, MMX, SIMD, AVX andSSE, or other operations, including specialized machine learninginstructions. In at least one embodiment, floating point ALU 1322 mayinclude, without limitation, a 64-bit by 64-bit floating point dividerto execute divide, square root, and remainder micro ops. In at least oneembodiment, instructions involving a floating point value may be handledwith floating point hardware. In at least one embodiment, ALU operationsmay be passed to fast ALUs 1316, 1318. In at least one embodiment, fastALUS 1316, 1318 may execute fast operations with an effective latency ofhalf a clock cycle. In at least one embodiment, most complex integeroperations go to slow ALU 1320 as slow ALU 1320 may include, withoutlimitation, integer execution hardware for long-latency type ofoperations, such as a multiplier, shifts, flag logic, and branchprocessing. In at least one embodiment, memory load/store operations maybe executed by AGUS 1312, 1314. In at least one embodiment, fast ALU1316, fast ALU 1318, and slow ALU 1320 may perform integer operations on64-bit data operands. In at least one embodiment, fast ALU 1316, fastALU 1318, and slow ALU 1320 may be implemented to support a variety ofdata bit sizes including sixteen, thirty-two, 128, 256, etc. In at leastone embodiment, floating point ALU 1322 and floating point move unit1324 may be implemented to support a range of operands having bits ofvarious widths. In at least one embodiment, floating point ALU 1322 andfloating point move unit 1324 may operate on 128-bit wide packed dataoperands in conjunction with SIMD and multimedia instructions.

In at least one embodiment, uop schedulers 1302, 1304, 1306, dispatchdependent operations before parent load has finished executing. In atleast one embodiment, as uops may be speculatively scheduled andexecuted in processor 1300, processor 1300 may also include logic tohandle memory misses. In at least one embodiment, if a data load missesin data cache, there may be dependent operations in flight in pipelinethat have left scheduler with temporarily incorrect data. In at leastone embodiment, a replay mechanism tracks and re-executes instructionsthat use incorrect data. In at least one embodiment, dependentoperations might need to be replayed and independent ones may be allowedto complete. In at least one embodiment, schedulers and replay mechanismof at least one embodiment of a processor may also be designed to catchinstruction sequences for text string comparison operations.

In at least one embodiment, registers may refer to on-board processorstorage locations that may be used as part of instructions to identifyoperands. In at least one embodiment, registers may be those that may beusable from outside of processor (from a programmer's perspective). Inat least one embodiment, registers might not be limited to a particulartype of circuit. Rather, in at least one embodiment, a register maystore data, provide data, and perform functions described herein. In atleast one embodiment, registers described herein may be implemented bycircuitry within a processor using any number of different techniques,such as dedicated physical registers, dynamically allocated physicalregisters using register renaming, combinations of dedicated anddynamically allocated physical registers, etc. In at least oneembodiment, integer registers store 32-bit integer data. A register fileof at least one embodiment also contains eight multimedia SIMD registersfor packed data.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodimentportions or all of inference and/or training logic 615 may beincorporated into execution block 1311 and other memory or registersshown or not shown. In at least one embodiment, in at least oneembodiment, training and/or inferencing techniques described herein mayuse one or more of ALUs illustrated in execution block 1311. Moreover,weight parameters may be stored in on-chip or off-chip memory and/orregisters (shown or not shown) that configure ALUs of execution block1311 to perform one or more machine learning algorithms, neural networkarchitectures, use cases, or training techniques described herein.

FIG. 14 illustrates a deep learning application processor 1400,according to at least one embodiment. In at least one embodiment, deeplearning application processor 1400 uses instructions that, if executedby deep learning application processor 1400, cause deep learningapplication processor 1400 to perform some or all of processes andtechniques described throughout this disclosure. In at least oneembodiment, deep learning application processor 1400 is anapplication-specific integrated circuit (ASIC). In at least oneembodiment, application processor 1400 performs matrix multiplyoperations either “hard-wired” into hardware as a result of performingone or more instructions or both. In at least one embodiment, deeplearning application processor 1400 includes, without limitation,processing clusters 1410(1)-1410(12), Inter-Chip Links (“ICLs”)1420(1)-1420(12), Inter-Chip Controllers (“ICCs”) 1430(1)-1430(2),memory controllers (“Mem Ctrlrs”) 1442(1)-1442(4), high bandwidth memoryphysical layer (“HBM PHY”) 1444(1)-1444(4), a management-controllercentral processing unit (“management-controller CPU”) 1450, a SerialPeripheral Interface, Inter-Integrated Circuit, and General PurposeInput/Output block (“SPI, I2C, GPIO”), a peripheral componentinterconnect express controller and direct memory access block (“PCIeController and DMA”) 1470, and a sixteen-lane peripheral componentinterconnect express port (“PCI Express x 16”) 1480.

In at least one embodiment, processing clusters 1410 may perform deeplearning operations, including inference or prediction operations basedon weight parameters calculated one or more training techniques,including those described herein. In at least one embodiment, eachprocessing cluster 1410 may include, without limitation, any number andtype of processors. In at least one embodiment, deep learningapplication processor 1400 may include any number and type of processingclusters 1400. In at least one embodiment, Inter-Chip Links 1420 arebi-directional. In at least one embodiment, Inter-Chip Links 1420 andInter-Chip Controllers 1430 enable multiple deep learning applicationprocessors 1400 to exchange information, including activationinformation resulting from performing one or more machine learningalgorithms embodied in one or more neural networks. In at least oneembodiment, deep learning application processor 1400 may include anynumber (including zero) and type of ICLs 1420 and ICCs 1430.

In at least one embodiment, HBM2s 1440 provide a total of 32 Gigabytes(GB) of memory. HBM2 1440(i) is associated with both memory controller1442(i) and HBM PHY 1444(i). In at least one embodiment, any number ofHBM2s 1440 may provide any type and total amount of high bandwidthmemory and may be associated with any number (including zero) and typeof memory controllers 1442 and HBM PHYs 1444. In at least oneembodiment, SPI, I2C, GPIO 1460, PCIe Controller and DMA 1470, and/orPCIe 1480 may be replaced with any number and type of blocks that enableany number and type of communication standards in any technicallyfeasible fashion.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment, deeplearning application processor 1400 is used to train a machine learningmodel, such as a neural network, to predict or infer informationprovided to deep learning application processor 1400. In at least oneembodiment, deep learning application processor 1400 is used to infer orpredict information based on a trained machine learning model (e.g.,neural network) that has been trained by another processor or system orby deep learning application processor 1400. In at least one embodiment,processor 1400 may be used to perform one or more neural network usecases described herein.

FIG. 15 is a block diagram of a neuromorphic processor 1500, accordingto at least one embodiment. In at least one embodiment, neuromorphicprocessor 1500 may receive one or more inputs from sources external toneuromorphic processor 1500. In at least one embodiment, these inputsmay be transmitted to one or more neurons 1502 within neuromorphicprocessor 1500. In at least one embodiment, neurons 1502 and componentsthereof may be implemented using circuitry or logic, including one ormore arithmetic logic units (ALUs). In at least one embodiment,neuromorphic processor 1500 may include, without limitation, thousandsor millions of instances of neurons 1502, but any suitable number ofneurons 1502 may be used. In at least one embodiment, each instance ofneuron 1502 may include a neuron input 1504 and a neuron output 1506. Inat least one embodiment, neurons 1502 may generate outputs that may betransmitted to inputs of other instances of neurons 1502. In at leastone embodiment, in at least one embodiment, neuron inputs 1504 andneuron outputs 1506 may be interconnected via synapses 1508.

In at least one embodiment, neurons 1502 and synapses 1508 may beinterconnected such that neuromorphic processor 1500 operates to processor analyze information received by neuromorphic processor 1500. In atleast one embodiment, neurons 1502 may transmit an output pulse (or“fire” or “spike”) when inputs received through neuron input 1504 exceeda threshold. In at least one embodiment, neurons 1502 may sum orintegrate signals received at neuron inputs 1504. In at least oneembodiment, in at least one embodiment, neurons 1502 may be implementedas leaky integrate-and-fire neurons, wherein if a sum (referred to as a“membrane potential”) exceeds a threshold value, neuron 1502 maygenerate an output (or “fire”) using a transfer function such as asigmoid or threshold function. In at least one embodiment, a leakyintegrate-and-fire neuron may sum signals received at neuron inputs 1504into a membrane potential and may also apply a decay factor (or leak) toreduce a membrane potential. In at least one embodiment, a leakyintegrate-and-fire neuron may fire if multiple input signals arereceived at neuron inputs 1504 rapidly enough to exceed a thresholdvalue (in at least one embodiment, this is before a membrane potentialdecays too low to fire). In at least one embodiment, neurons 1502 may beimplemented using circuits or logic that receive inputs, integrateinputs into a membrane potential, and decay a membrane potential. In atleast one embodiment, inputs may be averaged, or any other suitabletransfer function may be used. Furthermore, in at least one embodiment,neurons 1502 may include, without limitation, comparator circuits orlogic that generate an output spike at neuron output 1506 when result ofapplying a transfer function to neuron input 1504 exceeds a threshold.In at least one embodiment, once neuron 1502 fires, it may disregardpreviously received input information by, for example, resetting amembrane potential to 0 or another suitable default value. In at leastone embodiment, once membrane potential is reset to 0, neuron 1502 mayresume normal operation after a suitable period of time (or refractoryperiod).

In at least one embodiment, neurons 1502 may be interconnected throughsynapses 1508. In at least one embodiment, synapses 1508 may operate totransmit signals from an output of a first neuron 1502 to an input of asecond neuron 1502. In at least one embodiment, neurons 1502 maytransmit information over more than one instance of synapse 1508. In atleast one embodiment, one or more instances of neuron output 1506 may beconnected, via an instance of synapse 1508, to an instance of neuroninput 1504 in same neuron 1502. In at least one embodiment, an instanceof neuron 1502 generating an output to be transmitted over an instanceof synapse 1508 may be referred to as a “pre-synaptic neuron” withrespect to that instance of synapse 1508. In at least one embodiment, aninstance of neuron 1502 receiving an input transmitted over an instanceof synapse 1508 may be referred to as a “post-synaptic neuron” withrespect to that instance of synapse 1508. Because an instance of neuron1502 may receive inputs from one or more instances of synapse 1508, andmay also transmit outputs over one or more instances of synapse 1508, asingle instance of neuron 1502 may therefore be both a “pre-synapticneuron” and “post-synaptic neuron,” with respect to various instances ofsynapses 1508, in at least one embodiment.

In at least one embodiment, neurons 1502 may be organized into one ormore layers. Each instance of neuron 1502 may have one neuron output1506 that may fan out through one or more synapses 1508 to one or moreneuron inputs 1504. In at least one embodiment, neuron outputs 1506 ofneurons 1502 in a first layer 1510 may be connected to neuron inputs1504 of neurons 1502 in a second layer 1512. In at least one embodiment,layer 1510 may be referred to as a “feed-forward layer.” In at least oneembodiment, each instance of neuron 1502 in an instance of first layer1510 may fan out to each instance of neuron 1502 in second layer 1512.In at least one embodiment, first layer 1510 may be referred to as a“fully connected feed-forward layer.” In at least one embodiment, eachinstance of neuron 1502 in an instance of second layer 1512 may fan outto fewer than all instances of neuron 1502 in a third layer 1514. In atleast one embodiment, second layer 1512 may be referred to as a“sparsely connected feed-forward layer.” In at least one embodiment,neurons 1502 in second layer 1512 may fan out to neurons 1502 inmultiple other layers, including to neurons 1502 in (same) second layer1512. In at least one embodiment, second layer 1512 may be referred toas a “recurrent layer.” In at least one embodiment, neuromorphicprocessor 1500 may include, without limitation, any suitable combinationof recurrent layers and feed-forward layers, including, withoutlimitation, both sparsely connected feed-forward layers and fullyconnected feed-forward layers.

In at least one embodiment, neuromorphic processor 1500 may include,without limitation, a reconfigurable interconnect architecture ordedicated hard wired interconnects to connect synapse 1508 to neurons1502. In at least one embodiment, neuromorphic processor 1500 mayinclude, without limitation, circuitry or logic that allows synapses tobe allocated to different neurons 1502 as needed based on neural networktopology and neuron fan-in/out. In at least one embodiment, in at leastone embodiment, synapses 1508 may be connected to neurons 1502 using aninterconnect fabric, such as network-on-chip, or with dedicatedconnections. In at least one embodiment, synapse interconnections andcomponents thereof may be implemented using circuitry or logic.

FIG. 16A is a block diagram of a processing system, according to atleast one embodiment. In at least one embodiment, system 1600A includesone or more processors 1602 and one or more graphics processors 1608,and may be a single processor desktop system, a multiprocessorworkstation system, or a server system having a large number ofprocessors 1602 or processor cores 1607. In at least one embodiment,system 1600A is a processing platform incorporated within asystem-on-a-chip (SoC) integrated circuit for use in mobile, handheld,or embedded devices.

In at least one embodiment, system 1600A can include, or be incorporatedwithin a server-based gaming platform, a game console, including a gameand media console, a mobile gaming console, a handheld game console, oran online game console. In at least one embodiment, system 1600A is amobile phone, smart phone, tablet computing device or mobile Internetdevice. In at least one embodiment, processing system 1600A can alsoinclude, couple with, or be integrated within a wearable device, such asa smart watch wearable device, smart eyewear device, augmented realitydevice, or virtual reality device. In at least one embodiment,processing system 1600A is a television or set top box device having oneor more processors 1602 and a graphical interface generated by one ormore graphics processors 1608.

In at least one embodiment, one or more processors 1602 each include oneor more processor cores 1607 to process instructions which, whenexecuted, perform operations for system and user software. In at leastone embodiment, each of one or more processor cores 1607 is configuredto process a specific instruction set 1609. In at least one embodiment,instruction set 1609 may facilitate Complex Instruction Set Computing(CISC), Reduced Instruction Set Computing (RISC), or computing via aVery Long Instruction Word (VLIW). In at least one embodiment, processorcores 1607 may each process a different instruction set 1609, which mayinclude instructions to facilitate emulation of other instruction sets.In at least one embodiment, processor core 1607 may also include otherprocessing devices, such a Digital Signal Processor (DSP).

In at least one embodiment, processor 1602 includes cache memory 1604.In at least one embodiment, processor 1602 can have a single internalcache or multiple levels of internal cache. In at least one embodiment,cache memory is shared among various components of processor 1602. In atleast one embodiment, processor 1602 also uses an external cache (e.g.,a Level-3 (L3) cache or Last Level Cache (LLC)) (not shown), which maybe shared among processor cores 1607 using known cache coherencytechniques. In at least one embodiment, register file 1606 isadditionally included in processor 1602 which may include differenttypes of registers for storing different types of data (e.g., integerregisters, floating point registers, status registers, and aninstruction pointer register). In at least one embodiment, register file1606 may include general-purpose registers or other registers.

In at least one embodiment, one or more processor(s) 1602 are coupledwith one or more interface bus(es) 1610 to transmit communicationsignals such as address, data, or control signals between processor 1602and other components in system 1600A. In at least one embodiment,interface bus 1610, in one embodiment, can be a processor bus, such as aversion of a Direct Media Interface (DMI) bus. In at least oneembodiment, interface 1610 is not limited to a DMI bus, and may includeone or more Peripheral Component Interconnect buses (e.g., PCI, PCIExpress), memory busses, or other types of interface busses. In at leastone embodiment processor(s) 1602 include an integrated memory controller1616 and a platform controller hub 1630. In at least one embodiment,memory controller 1616 facilitates communication between a memory deviceand other components of system 1600A, while platform controller hub(PCH) 1630 provides connections to I/O devices via a local I/O bus.

In at least one embodiment, memory device 1620 can be a dynamic randomaccess memory (DRAM) device, a static random access memory (SRAM)device, flash memory device, phase-change memory device, or some othermemory device having suitable performance to serve as process memory. Inat least one embodiment memory device 1620 can operate as system memoryfor system 1600A, to store data 1622 and instructions 1621 for use whenone or more processors 1602 executes an application or process. In atleast one embodiment, memory controller 1616 also couples with an, in atleast one embodiment, external graphics processor 1612, which maycommunicate with one or more graphics processors 1608 in processors 1602to perform graphics and media operations. In at least one embodiment, adisplay device 1611 can connect to processor(s) 1602. In at least oneembodiment display device 1611 can include one or more of an internaldisplay device, as in a mobile electronic device or a laptop device oran external display device attached via a display interface (e.g.,DisplayPort, etc.). In at least one embodiment, display device 1611 caninclude a head mounted display (HMD) such as a stereoscopic displaydevice for use in virtual reality (VR) applications or augmented reality(AR) applications.

In at least one embodiment, platform controller hub 1630 enablesperipherals to connect to memory device 1620 and processor 1602 via ahigh-speed I/O bus. In at least one embodiment, I/O peripherals include,but are not limited to, an audio controller 1646, a network controller1634, a firmware interface 1628, a wireless transceiver 1626, touchsensors 1625, a data storage device 1624 (e.g., hard disk drive, flashmemory, etc.). In at least one embodiment, data storage device 1624 canconnect via a storage interface (e.g., SATA) or via a peripheral bus,such as a Peripheral Component Interconnect bus (e.g., PCI, PCIExpress). In at least one embodiment, touch sensors 1625 can includetouch screen sensors, pressure sensors, or fingerprint sensors. In atleast one embodiment, wireless transceiver 1626 can be a Wi-Fitransceiver, a Bluetooth transceiver, or a mobile network transceiversuch as a 3G, 4G, or Long Term Evolution (LTE) transceiver. In at leastone embodiment, firmware interface 1628 enables communication withsystem firmware, and can be, for example, a unified extensible firmwareinterface (UEFI). In at least one embodiment, network controller 1634can enable a network connection to a wired network. In at least oneembodiment, a high-performance network controller (not shown) coupleswith interface bus 1610. In at least one embodiment, audio controller1646 is a multi-channel high definition audio controller. In at leastone embodiment, system 1600A includes a legacy I/O controller 1640 forcoupling legacy (e.g., Personal System 2 (PS/2)) devices to system. Inat least one embodiment, platform controller hub 1630 can also connectto one or more Universal Serial Bus (USB) controllers 1642 connect inputdevices, such as keyboard and mouse 1643 combinations, a camera 1644, orother USB input devices.

In at least one embodiment, an instance of memory controller 1616 andplatform controller hub 1630 may be integrated into a discreet externalgraphics processor, such as external graphics processor 1612. In atleast one embodiment, platform controller hub 1630 and/or memorycontroller 1616 may be external to one or more processor(s) 1602. In atleast one embodiment, in at least one embodiment, system 1600A caninclude an external memory controller 1616 and platform controller hub1630, which may be configured as a memory controller hub and peripheralcontroller hub within a system chipset that is in communication withprocessor(s) 1602.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodimentportions or all of inference and/or training logic 615 may beincorporated into graphics processor 1600A. In at least one embodiment,in at least one embodiment, training and/or inferencing techniquesdescribed herein may use one or more of ALUs embodied in graphicsprocessor 1612. Moreover, in at least one embodiment, inferencing and/ortraining operations described herein may be done using logic other thanlogic illustrated in FIG. 6B or 6C. In at least one embodiment, weightparameters may be stored in on-chip or off-chip memory and/or registers(shown or not shown) that configure ALUs of graphics processor 1600A toperform one or more machine learning algorithms, neural networkarchitectures, use cases, or training techniques described herein.

FIG. 16B is a block diagram of a processor 1600B having one or moreprocessor cores 1602A-1602N, an integrated memory controller 1614, andan integrated graphics processor 1608, according to at least oneembodiment. In at least one embodiment, processor 1600B can includeadditional cores up to and including additional core 1602N representedby dashed lined boxes. In at least one embodiment, each of processorcores 1602A-1602N includes one or more internal cache units 1604A-1604N.In at least one embodiment, each processor core also has access to oneor more shared cached units 1606.

In at least one embodiment, internal cache units 1604A-1604N and sharedcache units 1606 represent a cache memory hierarchy within processor1600B. In at least one embodiment, cache memory units 1604A-1604N mayinclude at least one level of instruction and data cache within eachprocessor core and one or more levels of shared mid-level cache, such asa Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache,where a highest level of cache before external memory is classified asan LLC. In at least one embodiment, cache coherency logic maintainscoherency between various cache units 1606 and 1604A-1604N.

In at least one embodiment, processor 1600B may also include a set ofone or more bus controller units 1616 and a system agent core 1610. Inat least one embodiment, one or more bus controller units 1616 manage aset of peripheral buses, such as one or more PCI or PCI express busses.In at least one embodiment, system agent core 1610 provides managementfunctionality for various processor components. In at least oneembodiment, system agent core 1610 includes one or more integratedmemory controllers 1614 to manage access to various external memorydevices (not shown).

In at least one embodiment, one or more of processor cores 1602A-1602Ninclude support for simultaneous multi-threading. In at least oneembodiment, system agent core 1610 includes components for coordinatingand operating cores 1602A-1602N during multi-threaded processing. In atleast one embodiment, system agent core 1610 may additionally include apower control unit (PCU), which includes logic and components toregulate one or more power states of processor cores 1602A-1602N andgraphics processor 1608.

In at least one embodiment, processor 1600B additionally includesgraphics processor 1608 to execute graphics processing operations. In atleast one embodiment, graphics processor 1608 couples with shared cacheunits 1606, and system agent core 1610, including one or more integratedmemory controllers 1614. In at least one embodiment, system agent core1610 also includes a display controller 1611 to drive graphics processoroutput to one or more coupled displays. In at least one embodiment,display controller 1611 may also be a separate module coupled withgraphics processor 1608 via at least one interconnect, or may beintegrated within graphics processor 1608.

In at least one embodiment, a ring based interconnect unit 1612 is usedto couple internal components of processor 1600B. In at least oneembodiment, an alternative interconnect unit may be used, such as apoint-to-point interconnect, a switched interconnect, or othertechniques. In at least one embodiment, graphics processor 1608 coupleswith ring interconnect 1612 via an I/O link 1613.

In at least one embodiment, I/O link 1613 represents at least one ofmultiple varieties of I/O interconnects, including an on package I/Ointerconnect which facilitates communication between various processorcomponents and a high-performance embedded memory module 1618, such asan eDRAM module. In at least one embodiment, each of processor cores1602A-1602N and graphics processor 1608 use embedded memory modules 1618as a shared Last Level Cache.

In at least one embodiment, processor cores 1602A-1602N are homogenouscores executing a common instruction set architecture. In at least oneembodiment, processor cores 1602A-1602N are heterogeneous in terms ofinstruction set architecture (ISA), where one or more of processor cores1602A-1602N execute a common instruction set, while one or more othercores of processor cores 1602A-16-02N executes a subset of a commoninstruction set or a different instruction set. In at least oneembodiment, processor cores 1602A-1602N are heterogeneous in terms ofmicroarchitecture, where one or more cores having a relatively higherpower consumption couple with one or more power cores having a lowerpower consumption. In at least one embodiment, processor 1600B can beimplemented on one or more chips or as an SoC integrated circuit.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodimentportions or all of inference and/or training logic 615 may beincorporated into processor 1600B. In at least one embodiment, in atleast one embodiment, training and/or inferencing techniques describedherein may use one or more of ALUs embodied in graphics processor 1612,graphics core(s) 1602A-1602N, or other components in FIG. 16. Moreover,in at least one embodiment, inferencing and/or training operationsdescribed herein may be done using logic other than logic illustrated inFIG. 6B or 6C. In at least one embodiment, weight parameters may bestored in on-chip or off-chip memory and/or registers (shown or notshown) that configure ALUs of graphics processor 1600B to perform one ormore machine learning algorithms, neural network architectures, usecases, or training techniques described herein.

FIG. 16C is a block diagram of hardware logic of a graphics processorcore 1600C, according to at least one embodiment described herein. In atleast one embodiment, graphics processor core 1600C is included within agraphics core array. In at least one embodiment, graphics processor core1600C, sometimes referred to as a core slice, can be one or multiplegraphics cores within a modular graphics processor. In at least oneembodiment, graphics processor core 1600C is exemplary of one graphicscore slice, and a graphics processor as described herein may includemultiple graphics core slices based on target power and performanceenvelopes. In at least one embodiment, each graphics core 1600C caninclude a fixed function block 1630 coupled with multiple sub-cores1601A-1601F, also referred to as sub-slices, that include modular blocksof general-purpose and fixed function logic.

In at least one embodiment, fixed function block 1630 includes ageometry/fixed function pipeline 1636 that can be shared by allsub-cores in graphics processor 1600C, for example, in lower performanceand/or lower power graphics processor implementations. In at least oneembodiment, geometry/fixed function pipeline 1636 includes a 3D fixedfunction pipeline, a video front-end unit, a thread spawner and threaddispatcher, and a unified return buffer manager, which manages unifiedreturn buffers.

In at least one embodiment fixed, function block 1630 also includes agraphics SoC interface 1637, a graphics microcontroller 1638, and amedia pipeline 1639. In at least one embodiment fixed, graphics SoCinterface 1637 provides an interface between graphics core 1600C andother processor cores within a system on a chip integrated circuit. Inat least one embodiment, graphics microcontroller 1638 is a programmablesub-processor that is configurable to manage various functions ofgraphics processor 1600C, including thread dispatch, scheduling, andpre-emption. In at least one embodiment, media pipeline 1639 includeslogic to facilitate decoding, encoding, pre-processing, and/orpost-processing of multimedia data, including image and video data. Inat least one embodiment, media pipeline 1639 implements media operationsvia requests to compute or sampling logic within sub-cores 1601-1601F.

In at least one embodiment, SoC interface 1637 enables graphics core1600C to communicate with general-purpose application processor cores(e.g., CPUs) and/or other components within an SoC, including memoryhierarchy elements such as a shared last level cache memory, system RAM,and/or embedded on-chip or on-package DRAM. In at least one embodiment,SoC interface 1637 can also enable communication with fixed functiondevices within an SoC, such as camera imaging pipelines, and enables useof and/or implements global memory atomics that may be shared betweengraphics core 1600C and CPUs within an SoC. In at least one embodiment,SoC interface 1637 can also implement power management controls forgraphics core 1600C and enable an interface between a clock domain ofgraphic core 1600C and other clock domains within an SoC. In at leastone embodiment, SoC interface 1637 enables receipt of command buffersfrom a command streamer and global thread dispatcher that are configuredto provide commands and instructions to each of one or more graphicscores within a graphics processor. In at least one embodiment, commandsand instructions can be dispatched to media pipeline 1639, when mediaoperations are to be performed, or a geometry and fixed functionpipeline (e.g., geometry and fixed function pipeline 1636, geometry andfixed function pipeline 1614) when graphics processing operations are tobe performed.

In at least one embodiment, graphics microcontroller 1638 can beconfigured to perform various scheduling and management tasks forgraphics core 1600C. In at least one embodiment, graphicsmicrocontroller 1638 can perform graphics and/or compute workloadscheduling on various graphics parallel engines within execution unit(EU) arrays 1602A-1602F, 1604A-1604F within sub-cores 1601A-1601F. In atleast one embodiment, host software executing on a CPU core of an SoCincluding graphics core 1600C can submit workloads one of multiplegraphic processor doorbells, which invokes a scheduling operation on anappropriate graphics engine. In at least one embodiment, schedulingoperations include determining which workload to run next, submitting aworkload to a command streamer, pre-empting existing workloads runningon an engine, monitoring progress of a workload, and notifying hostsoftware when a workload is complete. In at least one embodiment,graphics microcontroller 1638 can also facilitate low-power or idlestates for graphics core 1600C, providing graphics core 1600C with anability to save and restore registers within graphics core 1600C acrosslow-power state transitions independently from an operating systemand/or graphics driver software on a system.

In at least one embodiment, graphics core 1600C may have greater than orfewer than illustrated sub-cores 1601A-1601F, up to N modular sub-cores.For each set of N sub-cores, in at least one embodiment, graphics core1600C can also include shared function logic 1610, shared and/or cachememory 1612, a geometry/fixed function pipeline 1614, as well asadditional fixed function logic 1616 to accelerate various graphics andcompute processing operations. In at least one embodiment, sharedfunction logic 1610 can include logic units (e.g., sampler, math, and/orinter-thread communication logic) that can be shared by each N sub-coreswithin graphics core 1600C. In at least one embodiment fixed, sharedand/or cache memory 1612 can be a last-level cache for N sub-cores1601A-1601F within graphics core 1600C and can also serve as sharedmemory that is accessible by multiple sub-cores. In at least oneembodiment, geometry/fixed function pipeline 1614 can be includedinstead of geometry/fixed function pipeline 1636 within fixed functionblock 1630 and can include same or similar logic units.

In at least one embodiment, graphics core 1600C includes additionalfixed function logic 1616 that can include various fixed functionacceleration logic for use by graphics core 1600C. In at least oneembodiment, additional fixed function logic 1616 includes an additionalgeometry pipeline for use in position only shading. In position-onlyshading, at least two geometry pipelines exist, whereas in a fullgeometry pipeline within geometry/fixed function pipeline 1616, 1636,and a cull pipeline, which is an additional geometry pipeline which maybe included within additional fixed function logic 1616. In at least oneembodiment, cull pipeline is a trimmed down version of a full geometrypipeline. In at least one embodiment, a full pipeline and a cullpipeline can execute different instances of an application, eachinstance having a separate context. In at least one embodiment, positiononly shading can hide long cull runs of discarded triangles, enablingshading to be completed earlier in some instances. In at least oneembodiment, in at least one embodiment, cull pipeline logic withinadditional fixed function logic 1616 can execute position shaders inparallel with a main application and generates critical results fasterthan a full pipeline, as cull pipeline fetches and shades positionattribute of vertices, without performing rasterization and rendering ofpixels to a frame buffer. In at least one embodiment, cull pipeline canuse generated critical results to compute visibility information for alltriangles without regard to whether those triangles are culled. In atleast one embodiment, full pipeline (which in this instance may bereferred to as a replay pipeline) can consume visibility information toskip culled triangles to shade only visible triangles that are finallypassed to a rasterization phase.

In at least one embodiment, additional fixed function logic 1616 canalso include machine-learning acceleration logic, such as fixed functionmatrix multiplication logic, for implementations including optimizationsfor machine learning training or inferencing.

In at least one embodiment, within each graphics sub-core 1601A-1601Fincludes a set of execution resources that may be used to performgraphics, media, and compute operations in response to requests bygraphics pipeline, media pipeline, or shader programs. In at least oneembodiment, graphics sub-cores 1601A-1601F include multiple EU arrays1602A-1602F, 1604A-1604F, thread dispatch and inter-thread communication(TD/IC) logic 1603A-1603F, a 3D (e.g., texture) sampler 1605A-1605F, amedia sampler 1606A-1606F, a shader processor 1607A-1607F, and sharedlocal memory (SLM) 1608A-1608F. EU arrays 1602A-1602F, 1604A-1604F eachinclude multiple execution units, which are general-purpose graphicsprocessing units capable of performing floating-point andinteger/fixed-point logic operations in service of a graphics, media, orcompute operation, including graphics, media, or compute shaderprograms. In at least one embodiment, TD/IC logic 1603A-1603F performslocal thread dispatch and thread control operations for execution unitswithin a sub-core and facilitate communication between threads executingon execution units of a sub-core. In at least one embodiment, 3D sampler1605A-1605F can read texture or other 3D graphics related data intomemory. In at least one embodiment, 3D sampler can read texture datadifferently based on a configured sample state and texture formatassociated with a given texture. In at least one embodiment, mediasampler 1606A-1606F can perform similar read operations based on a typeand format associated with media data. In at least one embodiment, eachgraphics sub-core 1601A-1601F can alternately include a unified 3D andmedia sampler. In at least one embodiment, threads executing onexecution units within each of sub-cores 1601A-1601F can make use ofshared local memory 1608A-1608F within each sub-core, to enable threadsexecuting within a thread group to execute using a common pool ofon-chip memory.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,portions or all of inference and/or training logic 615 may beincorporated into graphics processor 1610. In at least one embodiment,in at least one embodiment, training and/or inferencing techniquesdescribed herein may use one or more of ALUs embodied in graphicsprocessor 1612, graphics microcontroller 1638, geometry & fixed functionpipeline 1614 and 1636, or other logic in FIG. 16B. Moreover, in atleast one embodiment, inferencing and/or training operations describedherein may be done using logic other than logic illustrated in FIG. 6Bor 6C. In at least one embodiment, weight parameters may be stored inon-chip or off-chip memory and/or registers (shown or not shown) thatconfigure ALUs of graphics processor 1600C to perform one or moremachine learning algorithms, neural network architectures, use cases, ortraining techniques described herein.

FIGS. 16D-16E illustrate thread execution logic 1600D including an arrayof processing elements of a graphics processor core according to atleast one embodiment. FIG. 16D illustrates at least one embodiment, inwhich thread execution logic 1600D is used. FIG. 16E illustratesexemplary internal details of an execution unit, according to at leastone embodiment.

As illustrated in FIG. 16D, in at least one embodiment, thread executionlogic 1600D includes a shader processor 1602, a thread dispatcher 1604,instruction cache 1606, a scalable execution unit array including aplurality of execution units 1608A-1608N, sampler(s) 1610, a data cache1612, and a data port 1614. In at least one embodiment a scalableexecution unit array can dynamically scale by enabling or disabling oneor more execution units (e.g., any of execution unit 1608A, 1608B,1608C, 1608D, through 1608N-1 and 1608N) based on computationalrequirements of a workload, for example. In at least one embodiment,scalable execution units are interconnected via an interconnect fabricthat links to each of execution unit. In at least one embodiment, threadexecution logic 1600D includes one or more connections to memory, suchas system memory or cache memory, through one or more of instructioncache 1606, data port 1614, sampler 1610, and execution units1608A-1608N. In at least one embodiment, each execution unit (e.g.,1608A) is a stand-alone programmable general-purpose computational unitthat is capable of executing multiple simultaneous hardware threadswhile processing multiple data elements in parallel for each thread. Inat least one embodiment, array of execution units 1608A-1608N isscalable to include any number individual execution units.

In at least one embodiment, execution units 1608A-1608N are primarilyused to execute shader programs. In at least one embodiment, shaderprocessor 1602 can process various shader programs and dispatchexecution threads associated with shader programs via a threaddispatcher 1604. In at least one embodiment, thread dispatcher 1604includes logic to arbitrate thread initiation requests from graphics andmedia pipelines and instantiate requested threads on one or moreexecution units in execution units 1608A-1608N. In at least oneembodiment, in at least one embodiment, a geometry pipeline can dispatchvertex, tessellation, or geometry shaders to thread execution logic forprocessing. In at least one embodiment, thread dispatcher 1604 can alsoprocess runtime thread spawning requests from executing shader programs.

In at least one embodiment, execution units 1608A-1608N support aninstruction set that includes native support for many standard 3Dgraphics shader instructions, such that shader programs from graphicslibraries (e.g., Direct 3D and OpenGL) are executed with a minimaltranslation. In at least one embodiment, execution units support vertexand geometry processing (e.g., vertex programs, geometry programs,vertex shaders), pixel processing (e.g., pixel shaders, fragmentshaders) and general-purpose processing (e.g., compute and mediashaders). In at least one embodiment, each of execution units1608A-1608N, which include one or more arithmetic logic units (ALUs), iscapable of multi-issue single instruction multiple data (SIMD) executionand multi-threaded operation enables an efficient execution environmentdespite higher latency memory accesses. In at least one embodiment, eachhardware thread within each execution unit has a dedicatedhigh-bandwidth register file and associated independent thread-state. Inat least one embodiment, execution is multi-issue per clock to pipelinescapable of integer, single and double precision floating pointoperations, SIMD branch capability, logical operations, transcendentaloperations, and other miscellaneous operations. In at least oneembodiment, while waiting for data from memory or one of sharedfunctions, dependency logic within execution units 1608A-1608N causes awaiting thread to sleep until requested data has been returned. In atleast one embodiment, while a waiting thread is sleeping, hardwareresources may be devoted to processing other threads. In at least oneembodiment, in at least one embodiment, during a delay associated with avertex shader operation, an execution unit can perform operations for apixel shader, fragment shader, or another type of shader program,including a different vertex shader.

In at least one embodiment, each execution unit in execution units1608A-1608N operates on arrays of data elements. In at least oneembodiment, a number of data elements is “execution size,” or number ofchannels for an instruction. In at least one embodiment, an executionchannel is a logical unit of execution for data element access, masking,and flow control within instructions. In at least one embodiment, anumber of channels may be independent of a number of physical ArithmeticLogic Units (ALUs) or Floating Point Units (FPUs) for a particulargraphics processor. In at least one embodiment, execution units1608A-1608N support integer and floating-point data types.

In at least one embodiment, an execution unit instruction set includesSIMD instructions. In at least one embodiment, various data elements canbe stored as a packed data type in a register and an execution unit willprocess various elements based on data size of elements. In at least oneembodiment, in at least one embodiment, when operating on a 256-bit widevector, 256 bits of a vector are stored in a register and an executionunit operates on a vector as four separate 64-bit packed data elements(Quad-Word (QW) size data elements), eight separate 32-bit packed dataelements (Double Word (DW) size data elements), sixteen separate 16-bitpacked data elements (Word (W) size data elements), or thirty-twoseparate 8-bit data elements (byte (B) size data elements). However, inat least one embodiment, different vector widths and register sizes arepossible.

In at least one embodiment, one or more execution units can be combinedinto a fused execution unit 1609A-1609N having thread control logic(1607A-1607N) that is common to fused EUs. In at least one embodiment,multiple EUs can be fused into an EU group. In at least one embodiment,each EU in fused EU group can be configured to execute a separate SIMDhardware thread. Number of EUs in a fused EU group can vary according tovarious embodiments. In at least one embodiment, various SIMD widths canbe performed per-EU, including but not limited to SIMD8, SIMD16, andSIMD32. In at least one embodiment, each fused graphics execution unit1609A-1609N includes at least two execution units. In at least oneembodiment, in at least one embodiment, fused execution unit 1609Aincludes a first EU 1608A, second EU 1608B, and thread control logic1607A that is common to first EU 1608A and second EU 1608B. In at leastone embodiment, thread control logic 1607A controls threads executed onfused graphics execution unit 1609A, allowing each EU within fusedexecution units 1609A-1609N to execute using a common instructionpointer register.

In at least one embodiment, one or more internal instruction caches(e.g., 1606) are included in thread execution logic 1600D to cachethread instructions for execution units. In at least one embodiment, oneor more data caches (e.g., 1612) are included to cache thread dataduring thread execution. In at least one embodiment, a sampler 1610 isincluded to provide texture sampling for 3D operations and mediasampling for media operations. In at least one embodiment, sampler 1610includes specialized texture or media sampling functionality to processtexture or media data during a sampling process before providing sampleddata to an execution unit.

During execution, in at least one embodiment, graphics and mediapipelines send thread initiation requests to thread execution logic1600D via thread spawning and dispatch logic. In at least oneembodiment, once a group of geometric objects has been processed andrasterized into pixel data, pixel processor logic (e.g., pixel shaderlogic, fragment shader logic, etc.) within shader processor 1602 isinvoked to further compute output information and cause results to bewritten to output surfaces (e.g., color buffers, depth buffers, stencilbuffers, etc.). In at least one embodiment, a pixel shader or fragmentshader calculates values of various vertex attributes that are to beinterpolated across a rasterized object. In at least one embodiment,pixel processor logic within shader processor 1602 then executes anapplication programming interface (API)-supplied pixel or fragmentshader program. In at least one embodiment, to execute a shader program,shader processor 1602 dispatches threads to an execution unit (e.g.,1608A) via thread dispatcher 1604. In at least one embodiment, shaderprocessor 1602 uses texture sampling logic in sampler 1610 to accesstexture data in texture maps stored in memory. In at least oneembodiment, arithmetic operations on texture data and input geometrydata compute pixel color data for each geometric fragment, or discardsone or more pixels from further processing.

In at least one embodiment, data port 1614 provides a memory accessmechanism for thread execution logic 1600D to output processed data tomemory for further processing on a graphics processor output pipeline.In at least one embodiment, data port 1614 includes or couples to one ormore cache memories (e.g., data cache 1612) to cache data for memoryaccess via a data port.

As illustrated in FIG. 16E, in at least one embodiment, a graphicsexecution unit 1608 can include an instruction fetch unit 1637, ageneral register file array (GRF) 1624, an architectural register filearray (ARF) 1626, a thread arbiter 1622, a send unit 1630, a branch unit1632, a set of SIMD floating point units (FPUs) 1634, and, in at leastone embodiment, a set of dedicated integer SIMD ALUs 1635. In at leastone embodiment, GRF 1624 and ARF 1626 includes a set of general registerfiles and architecture register files associated with each simultaneoushardware thread that may be active in graphics execution unit 1608. Inat least one embodiment, per thread architectural state is maintained inARF 1626, while data used during thread execution is stored in GRF 1624.In at least one embodiment, execution state of each thread, includinginstruction pointers for each thread, can be held in thread-specificregisters in ARF 1626.

In at least one embodiment, graphics execution unit 1608 has anarchitecture that is a combination of Simultaneous Multi-Threading (SMT)and fine-grained Interleaved Multi-Threading (IMT). In at least oneembodiment, architecture has a modular configuration that can befine-tuned at design time based on a target number of simultaneousthreads and number of registers per execution unit, where execution unitresources are divided across logic used to execute multiple simultaneousthreads.

In at least one embodiment, graphics execution unit 1608 can co-issuemultiple instructions, which may each be different instructions. In atleast one embodiment, thread arbiter 1622 of graphics execution unitthread 1608 can dispatch instructions to one of send unit 1630, branchunit 1642, or SIMD FPU(s) 1634 for execution. In at least oneembodiment, each execution thread can access 128 general-purposeregisters within GRF 1624, where each register can store 32 bytes,accessible as a SIMD 8-element vector of 32-bit data elements. In atleast one embodiment, each execution unit thread has access to 4 Kbyteswithin GRF 1624, although embodiments are not so limited, and greater orfewer register resources may be provided in other embodiments. In atleast one embodiment, up to seven threads can execute simultaneously,although a number of threads per execution unit can also vary accordingto embodiments. In at least one embodiment, in which seven threads mayaccess 4 Kbytes, GRF 1624 can store a total of 28 Kbytes. In at leastone embodiment, flexible addressing modes can permit registers to beaddressed together to build effectively wider registers or to representstrided rectangular block data structures.

In at least one embodiment, memory operations, sampler operations, andother longer-latency system communications are dispatched via “send”instructions that are executed by message passing send unit 1630. In atleast one embodiment, branch instructions are dispatched to a dedicatedbranch unit 1632 to facilitate SIMD divergence and eventual convergence.

In at least one embodiment graphics execution unit 1608 includes one ormore SIMD floating point units (FPU(s)) 1634 to perform floating-pointoperations. In at least one embodiment, FPU(s) 1634 also support integercomputation. In at least one embodiment FPU(s) 1634 can SIMD execute upto M number of 32-bit floating-point (or integer) operations, or SIMDexecute up to 2M 16-bit integer or 16-bit floating-point operations. Inat least one embodiment, at least one of FPU(s) provides extended mathcapability to support high-throughput transcendental math functions anddouble precision 64-bit floating-point. In at least one embodiment, aset of 8-bit integer SIMD ALUs 1635 are also present, and may bespecifically optimized to perform operations associated with machinelearning computations.

In at least one embodiment, arrays of multiple instances of graphicsexecution unit 1608 can be instantiated in a graphics sub-core grouping(e.g., a sub-slice). In at least one embodiment, execution unit 1608 canexecute instructions across a plurality of execution channels. In atleast one embodiment, each thread executed on graphics execution unit1608 is executed on a different channel.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment,portions or all of inference and/or training logic 615 may beincorporated into execution logic 1600D. Moreover, in at least oneembodiment, inferencing and/or training operations described herein maybe done using logic other than logic illustrated in FIG. 6B or 6C. In atleast one embodiment, weight parameters may be stored in on-chip oroff-chip memory and/or registers (shown or not shown) that configureALUs of execution logic 1600D to perform one or more machine learningalgorithms, neural network architectures, use cases, or trainingtechniques described herein.

FIG. 17A illustrates a parallel processing unit (“PPU”) 1700A, accordingto at least one embodiment. In at least one embodiment, PPU 1700A isconfigured with machine-readable code that, if executed by PPU 1700A,causes PPU 1700A to perform some or all of processes and techniquesdescribed throughout this disclosure. In at least one embodiment, PPU1700A is a multi-threaded processor that is implemented on one or moreintegrated circuit devices and that utilizes multithreading as alatency-hiding technique designed to process computer-readableinstructions (also referred to as machine-readable instructions orsimply instructions) on multiple threads in parallel. In at least oneembodiment, a thread refers to a thread of execution and is aninstantiation of a set of instructions configured to be executed by PPU1700A. In at least one embodiment, PPU 1700A is a graphics processingunit (“GPU”) configured to implement a graphics rendering pipeline forprocessing three-dimensional (“3D”) graphics data in order to generatetwo-dimensional (“2D”) image data for display on a display device suchas a liquid crystal display (“LCD”) device. In at least one embodiment,PPU 1700A is utilized to perform computations such as linear algebraoperations and machine-learning operations. FIG. 17A illustrates anexample parallel processor for illustrative purposes only and should beconstrued as a non-limiting example of processor architecturescontemplated within scope of this disclosure and that any suitableprocessor may be employed to supplement and/or substitute for same.

In at least one embodiment, one or more PPUs 1700A are configured toaccelerate High Performance Computing (“HPC”), datacenter, and machinelearning applications. In at least one embodiment, PPU 1700A isconfigured to accelerate deep learning systems and applicationsincluding following non-limiting examples: autonomous vehicle platforms,deep learning, high-accuracy speech, image, text recognition systems,intelligent video analytics, molecular simulations, drug discovery,disease diagnosis, weather forecasting, big data analytics, astronomy,molecular dynamics simulation, financial modeling, robotics, factoryautomation, real-time language translation, online search optimizations,and personalized user recommendations, and more.

In at least one embodiment, PPU 1700A includes, without limitation, anInput/Output (“I/O”) unit 1706, a front-end unit 1710, a scheduler unit1712, a work distribution unit 1714, a hub 1716, a crossbar (“Xbar”)1720, one or more general processing clusters (“GPCs”) 1718, and one ormore partition units (“memory partition units”) 1722. In at least oneembodiment, PPU 1700A is connected to a host processor or other PPUs1700A via one or more high-speed GPU interconnects (“GPU interconnects”)1708. In at least one embodiment, PPU 1700A is connected to a hostprocessor or other peripheral devices via an interconnect 1702. In atleast one embodiment, PPU 1700A is connected to a local memorycomprising one or more memory devices (“memory”) 1704. In at least oneembodiment, memory devices 1704 include, without limitation, one or moredynamic random access memory (“DRAM”) devices. In at least oneembodiment, one or more DRAM devices are configured and/or configurableas high-bandwidth memory (“HBM”) subsystems, with multiple DRAM diesstacked within each device.

In at least one embodiment, high-speed GPU interconnect 1708 may referto a wire-based multi-lane communications link that is used by systemsto scale and include one or more PPUs 1700A combined with one or morecentral processing units (“CPUs”), supports cache coherence between PPUs1700A and CPUs, and CPU mastering. In at least one embodiment, dataand/or commands are transmitted by high-speed GPU interconnect 1708through hub 1716 to/from other units of PPU 1700A such as one or morecopy engines, video encoders, video decoders, power management units,and other components which may not be explicitly illustrated in FIG.17A.

In at least one embodiment, I/O unit 1706 is configured to transmit andreceive communications (e.g., commands, data) from a host processor (notillustrated in FIG. 17A) over system bus 1702. In at least oneembodiment, I/O unit 1706 communicates with host processor directly viasystem bus 1702 or through one or more intermediate devices such as amemory bridge. In at least one embodiment, I/O unit 1706 may communicatewith one or more other processors, such as one or more of PPUs 1700A viasystem bus 1702. In at least one embodiment, I/O unit 1706 implements aPeripheral Component Interconnect Express (“PCIe”) interface forcommunications over a PCIe bus. In at least one embodiment, I/O unit1706 implements interfaces for communicating with external devices.

In at least one embodiment, I/O unit 1706 decodes packets received viasystem bus 1702. In at least one embodiment, at least some packetsrepresent commands configured to cause PPU 1700A to perform variousoperations. In at least one embodiment, I/O unit 1706 transmits decodedcommands to various other units of PPU 1700A as specified by commands.In at least one embodiment, commands are transmitted to front-end unit1710 and/or transmitted to hub 1716 or other units of PPU 1700A such asone or more copy engines, a video encoder, a video decoder, a powermanagement unit, etc. (not explicitly illustrated in FIG. 17A). In atleast one embodiment, I/O unit 1706 is configured to routecommunications between and among various logical units of PPU 1700A.

In at least one embodiment, a program executed by host processor encodesa command stream in a buffer that provides workloads to PPU 1700A forprocessing. In at least one embodiment, a workload comprisesinstructions and data to be processed by those instructions. In at leastone embodiment, buffer is a region in a memory that is accessible (e.g.,read/write) by both host processor and PPU 1700A—a host interface unitmay be configured to access buffer in a system memory connected tosystem bus 1702 via memory requests transmitted over system bus 1702 byI/O unit 1706. In at least one embodiment, host processor writes commandstream to buffer and then transmits a pointer to start of command streamto PPU 1700A such that front-end unit 1710 receives pointers to one ormore command streams and manages one or more command streams, readingcommands from command streams and forwarding commands to various unitsof PPU 1700A.

In at least one embodiment, front-end unit 1710 is coupled to schedulerunit 1712 that configures various GPCs 1718 to process tasks defined byone or more command streams. In at least one embodiment, scheduler unit1712 is configured to track state information related to various tasksmanaged by scheduler unit 1712 where state information may indicatewhich of GPCs 1718 a task is assigned to, whether task is active orinactive, a priority level associated with task, and so forth. In atleast one embodiment, scheduler unit 1712 manages execution of aplurality of tasks on one or more of GPCs 1718.

In at least one embodiment, scheduler unit 1712 is coupled to workdistribution unit 1714 that is configured to dispatch tasks forexecution on GPCs 1718. In at least one embodiment, work distributionunit 1714 tracks a number of scheduled tasks received from schedulerunit 1712 and work distribution unit 1714 manages a pending task pooland an active task pool for each of GPCs 1718. In at least oneembodiment, pending task pool comprises a number of slots (e.g., 32slots) that contain tasks assigned to be processed by a particular GPC1718; active task pool may comprise a number of slots (e.g., 4 slots)for tasks that are actively being processed by GPCs 1718 such that asone of GPCs 1718 completes execution of a task, that task is evictedfrom active task pool for GPC 1718 and one of other tasks from pendingtask pool is selected and scheduled for execution on GPC 1718. In atleast one embodiment, if an active task is idle on GPC 1718, such aswhile waiting for a data dependency to be resolved, then active task isevicted from GPC 1718 and returned to pending task pool while anothertask in pending task pool is selected and scheduled for execution on GPC1718.

In at least one embodiment, work distribution unit 1714 communicateswith one or more GPCs 1718 via XBar 1720. In at least one embodiment,XBar 1720 is an interconnect network that couples many of units of PPU1700A to other units of PPU 1700A and can be configured to couple workdistribution unit 1714 to a particular GPC 1718. In at least oneembodiment, one or more other units of PPU 1700A may also be connectedto XBar 1720 via hub 1716.

In at least one embodiment, tasks are managed by scheduler unit 1712 anddispatched to one of GPCs 1718 by work distribution unit 1714. GPC 1718is configured to process task and generate results. In at least oneembodiment, results may be consumed by other tasks within GPC 1718,routed to a different GPC 1718 via XBar 1720, or stored in memory 1704.In at least one embodiment, results can be written to memory 1704 viapartition units 1722, which implement a memory interface for reading andwriting data to/from memory 1704. In at least one embodiment, resultscan be transmitted to another PPU 1704 or CPU via high-speed GPUinterconnect 1708. In at least one embodiment, PPU 1700A includes,without limitation, a number U of partition units 1722 that is equal tonumber of separate and distinct memory devices 1704 coupled to PPU1700A. In at least one embodiment, partition unit 1722 will be describedin more detail below in conjunction with FIG. 17C.

In at least one embodiment, a host processor executes a driver kernelthat implements an application programming interface (“API”) thatenables one or more applications executing on host processor to scheduleoperations for execution on PPU 1700A. In at least one embodiment,multiple compute applications are simultaneously executed by PPU 1700Aand PPU 1700A provides isolation, quality of service (“QoS”), andindependent address spaces for multiple compute applications. In atleast one embodiment, an application generates instructions (e.g., inform of API calls) that cause driver kernel to generate one or moretasks for execution by PPU 1700A and driver kernel outputs tasks to oneor more streams being processed by PPU 1700A. In at least oneembodiment, each task comprises one or more groups of related threads,which may be referred to as a warp. In at least one embodiment, a warpcomprises a plurality of related threads (e.g., 32 threads) that can beexecuted in parallel. In at least one embodiment, cooperating threadscan refer to a plurality of threads including instructions to performtask and that exchange data through shared memory. In at least oneembodiment, threads and cooperating threads are described in moredetail, in accordance with at least one embodiment, in conjunction withFIG. 17C.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment, deeplearning application processor is used to train a machine learningmodel, such as a neural network, to predict or infer informationprovided to PPU 1700A. In at least one embodiment, PPU 1700A is used toinfer or predict information based on a trained machine learning model(e.g., neural network) that has been trained by another processor orsystem or by PPU 1700A. In at least one embodiment, PPU 1700A may beused to perform one or more neural network use cases described herein.

FIG. 17B illustrates a general processing cluster (“GPC”) 1700B,according to at least one embodiment. In at least one embodiment, GPC1700B is GPC 1718 of FIG. 17A. In at least one embodiment, each GPC1700B includes, without limitation, a number of hardware units forprocessing tasks and each GPC 1700B includes, without limitation, apipeline manager 1702, a pre-raster operations unit (“PROP”) 1704, araster engine 1708, a work distribution crossbar (“WDX”) 1716, a memorymanagement unit (“MMU”) 1718, one or more Data Processing Clusters(“DPCs”) 1706, and any suitable combination of parts.

In at least one embodiment, operation of GPC 1700B is controlled bypipeline manager 1702. In at least one embodiment, pipeline manager 1702manages configuration of one or more DPCs 1706 for processing tasksallocated to GPC 1700B. In at least one embodiment, pipeline manager1702 configures at least one of one or more DPCs 1706 to implement atleast a portion of a graphics rendering pipeline. In at least oneembodiment, DPC 1706 is configured to execute a vertex shader program ona programmable streaming multi-processor (“SM”) 1714. In at least oneembodiment, pipeline manager 1702 is configured to route packetsreceived from a work distribution unit to appropriate logical unitswithin GPC 1700B, in at least one embodiment, and some packets may berouted to fixed function hardware units in PROP 1704 and/or rasterengine 1708 while other packets may be routed to DPCs 1706 forprocessing by a primitive engine 1712 or SM 1714. In at least oneembodiment, pipeline manager 1702 configures at least one of DPCs 1706to implement a neural network model and/or a computing pipeline.

In at least one embodiment, PROP unit 1704 is configured, in at leastone embodiment, to route data generated by raster engine 1708 and DPCs1706 to a Raster Operations (“ROP”) unit in partition unit 1722,described in more detail above in conjunction with FIG. 17A. In at leastone embodiment, PROP unit 1704 is configured to perform optimizationsfor color blending, organize pixel data, perform address translations,and more. In at least one embodiment, raster engine 1708 includes,without limitation, a number of fixed function hardware units configuredto perform various raster operations, in at least one embodiment, andraster engine 1708 includes, without limitation, a setup engine, acoarse raster engine, a culling engine, a clipping engine, a fine rasterengine, a tile coalescing engine, and any suitable combination thereof.In at least one embodiment, setup engine receives transformed verticesand generates plane equations associated with geometric primitivedefined by vertices; plane equations are transmitted to coarse rasterengine to generate coverage information (e.g., an x, y coverage mask fora tile) for primitive; output of coarse raster engine is transmitted toculling engine where fragments associated with primitive that fail az-test are culled, and transmitted to a clipping engine where fragmentslying outside a viewing frustum are clipped. In at least one embodiment,fragments that survive clipping and culling are passed to fine rasterengine to generate attributes for pixel fragments based on planeequations generated by setup engine. In at least one embodiment, outputof raster engine 1708 comprises fragments to be processed by anysuitable entity such as by a fragment shader implemented within DPC1706.

In at least one embodiment, each DPC 1706 included in GPC 1700Bcomprise, without limitation, an M-Pipe Controller (“MPC”) 1710;primitive engine 1712; one or more SMs 1714; and any suitablecombination thereof. In at least one embodiment, MPC 1710 controlsoperation of DPC 1706, routing packets received from pipeline manager1702 to appropriate units in DPC 1706. In at least one embodiment,packets associated with a vertex are routed to primitive engine 1712,which is configured to fetch vertex attributes associated with vertexfrom memory; in contrast, packets associated with a shader program maybe transmitted to SM 1714.

In at least one embodiment, SM 1714 comprises, without limitation, aprogrammable streaming processor that is configured to process tasksrepresented by a number of threads. In at least one embodiment, SM 1714is multi-threaded and configured to execute a plurality of threads(e.g., 32 threads) from a particular group of threads concurrently andimplements a Single-Instruction, Multiple-Data (“SIMD”) architecturewhere each thread in a group of threads (e.g., a warp) is configured toprocess a different set of data based on same set of instructions. In atleast one embodiment, all threads in group of threads execute sameinstructions. In at least one embodiment, SM 1714 implements aSingle-Instruction, Multiple Thread (“SIMT”) architecture wherein eachthread in a group of threads is configured to process a different set ofdata based on same set of instructions, but where individual threads ingroup of threads are allowed to diverge during execution. In at leastone embodiment, a program counter, call stack, and execution state ismaintained for each warp, enabling concurrency between warps and serialexecution within warps when threads within warp diverge. In anotherembodiment, a program counter, call stack, and execution state ismaintained for each individual thread, enabling equal concurrencybetween all threads, within and between warps. In at least oneembodiment, execution state is maintained for each individual thread andthreads executing same instructions may be converged and executed inparallel for better efficiency. At least one embodiment of SM 1714 aredescribed in more detail below.

In at least one embodiment, MMU 1718 provides an interface between GPC1700B and memory partition unit (e.g., partition unit 1722 of FIG. 17A)and MMU 1718 provides translation of virtual addresses into physicaladdresses, memory protection, and arbitration of memory requests. In atleast one embodiment, MMU 1718 provides one or more translationlookaside buffers (“TLBs”) for performing translation of virtualaddresses into physical addresses in memory.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment, deeplearning application processor is used to train a machine learningmodel, such as a neural network, to predict or infer informationprovided to GPC 1700B. In at least one embodiment, GPC 1700B is used toinfer or predict information based on a trained machine learning model(e.g., neural network) that has been trained by another processor orsystem or by GPC 1700B. In at least one embodiment, GPC 1700B may beused to perform one or more neural network use cases described herein.

FIG. 17C illustrates a memory partition unit 1700C of a parallelprocessing unit (“PPU”), in accordance with at least one embodiment. Inat least one embodiment, memory partition unit 1700C includes, withoutlimitation, a Raster Operations (“ROP”) unit 1702; a level two (“L2”)cache 1704; a memory interface 1706; and any suitable combinationthereof. In at least one embodiment, memory interface 1706 is coupled tomemory. In at least one embodiment, memory interface 1706 may implement32, 64, 128, 1024-bit data buses, or similar implementations, forhigh-speed data transfer. In at least one embodiment, PPU incorporates Umemory interfaces 1706, one memory interface 1706 per pair of partitionunits 1700C, where each pair of partition units 1700C is connected to acorresponding memory device. In at least one embodiment, in at least oneembodiment, PPU may be connected to up to Y memory devices, such as highbandwidth memory stacks or graphics double-data-rate, version 5,synchronous dynamic random al7ess memory (“GDDR5 SDRAM”).

In at least one embodiment, memory interface 1706 implements a highbandwidth memory second generation (“HBM2”) memory interface and Yequals half U. In at least one embodiment, HBM2 memory stacks arelocated on same physical package as PPU, providing substantial power andarea savings compared with GDDR5 SDRAM systems. In at least oneembodiment, each HBM2 stack includes, without limitation, four memorydies and Y equals 4, with each HBM2 stack including two 128-bit channelsper die for a total of 8 channels and a data bus width of 1024 bits. Inat least one embodiment, memory supports Single-Error CorrectingDouble-Error Detecting (“SECDED”) Error Correction Code (“ECC”) toprotect data. In at least one embodiment, ECC provides higherreliability for compute applications that are sensitive to datacorruption.

In at least one embodiment, PPU implements a multi-level memoryhierarchy. In at least one embodiment, memory partition unit 1700Csupports a unified memory to provide a single unified virtual addressspace for central processing unit (“CPU”) and PPU memory, enabling datasharing between virtual memory systems. In at least one embodiment,frequency of accesses by a PPU to memory located on other processors istraced to ensure that memory pages are moved to physical memory of PPUthat is accessing pages more frequently. In at least one embodiment,high-speed GPU interconnect 1708 supports address translation servicesallowing PPU to directly access a CPU's page tables and providing fullaccess to CPU memory by PPU.

In at least one embodiment, copy engines transfer data between multiplePPUs or between PPUs and CPUs. In at least one embodiment, copy enginescan generate page faults for addresses that are not mapped into pagetables and memory partition unit 1700C then services page faults,mapping addresses into page table, after which copy engine performstransfer. In at least one embodiment, memory is pinned (in at least oneembodiment, non-pageable) for multiple copy engine operations betweenmultiple processors, substantially reducing available memory. In atleast one embodiment, with hardware page faulting, addresses can bepassed to copy engines without regard as to whether memory pages areresident, and copy process is transparent.

Data from memory 1704 of FIG. 17A or other system memory is fetched bymemory partition unit 1700C and stored in L2 cache 1704, which islocated on-chip and is shared between various GPCs, in accordance withat least one embodiment. Each memory partition unit 1700C, in at leastone embodiment, includes, without limitation, at least a portion of L2cache associated with a corresponding memory device. In at least oneembodiment, lower level caches are implemented in various units withinGPCs. In at least one embodiment, each of SMs 1714 may implement a levelone (“L1”) cache wherein L1 cache is private memory that is dedicated toa particular SM 1714 and data from L2 cache 1704 is fetched and storedin each of L1 caches for processing in functional units of SMs 1714. Inat least one embodiment, L2 cache 1704 is coupled to memory interface1706 and XBar 1720.

ROP unit 1702 performs graphics raster operations related to pixelcolor, such as color compression, pixel blending, and more, in at leastone embodiment. ROP unit 1702, in at least one embodiment, implementsdepth testing in conjunction with raster engine 1708, receiving a depthfor a sample location associated with a pixel fragment from cullingengine of raster engine 1708. In at least one embodiment, depth istested against a corresponding depth in a depth buffer for a samplelocation associated with fragment. In at least one embodiment, iffragment passes depth test for sample location, then ROP unit 1702updates depth buffer and transmits a result of depth test to rasterengine 1708. It will be appreciated that number of partition units 1700Cmay be different than number of GPCs and, therefore, each ROP unit 1702can, in at least one embodiment, be coupled to each of GPCs. In at leastone embodiment, ROP unit 1702 tracks packets received from differentGPCs and determines which that a result generated by ROP unit 1702 isrouted to through XBar 1720.

FIG. 17D illustrates a streaming multi-processor (“SM”) 1700D, accordingto at least one embodiment. In at least one embodiment, SM 1700D is SM1714 of FIG. 17B. In at least one embodiment, SM 1700D includes, withoutlimitation, an instruction cache 1702; one or more scheduler units 1704;a register file 1708; one or more processing cores (“cores”) 1710; oneor more special function units (“SFUs”) 1712; one or more load/storeunits (“LSUs”) 1714; an interconnect network 1716; a shared memory/levelone (“L1”) cache 1718; and any suitable combination thereof. In at leastone embodiment, a work distribution unit dispatches tasks for executionon general processing clusters (“GPCs”) of parallel processing units(“PPUs”) and each task is allocated to a particular Data ProcessingCluster (“DPC”) within a GPC and, if task is associated with a shaderprogram, task is allocated to one of SMs 1700D. In at least oneembodiment, scheduler unit 1704 receives tasks from work distributionunit and manages instruction scheduling for one or more thread blocksassigned to SM 1700D. In at least one embodiment, scheduler unit 1704schedules thread blocks for execution as warps of parallel threads,wherein each thread block is allocated at least one warp. In at leastone embodiment, each warp executes threads. In at least one embodiment,scheduler unit 1704 manages a plurality of different thread blocks,allocating warps to different thread blocks and then dispatchinginstructions from plurality of different cooperative groups to variousfunctional units (e.g., processing cores 1710, SFUs 1712, and LSUs 1714)during each clock cycle.

In at least one embodiment, Cooperative Groups may refer to aprogramming model for organizing groups of communicating threads thatallows developers to express granularity at which threads arecommunicating, enabling expression of richer, more efficient paralleldecompositions. In at least one embodiment, cooperative launch APIssupport synchronization amongst thread blocks for execution of parallelalgorithms. In at least one embodiment, applications of programmingmodels provide a single, simple construct for synchronizing cooperatingthreads: a barrier across all threads of a thread block (e.g.,syncthreads( ) function). However, in at least one embodiment,programmers may define groups of threads at smaller than thread blockgranularities and synchronize within defined groups to enable greaterperformance, design flexibility, and software reuse in form ofcollective group-wide function interfaces. In at least one embodiment,Cooperative Groups enables programmers to define groups of threadsexplicitly at sub-block (in at least one embodiment, as small as asingle thread) and multi-block granularities, and to perform collectiveoperations such as synchronization on threads in a cooperative group. Inat least one embodiment, programming model supports clean compositionacross software boundaries, so that libraries and utility functions cansynchronize safely within their local context without having to makeassumptions about convergence. In at least one embodiment, CooperativeGroups primitives enable new patterns of cooperative parallelism,including, without limitation, producer-consumer parallelism,opportunistic parallelism, and global synchronization across an entiregrid of thread blocks.

In at least one embodiment, a dispatch unit 1706 is configured totransmit instructions to one or more of functional units and schedulerunit 1704 includes, without limitation, two dispatch units 1706 thatenable two different instructions from same warp to be dispatched duringeach clock cycle. In at least one embodiment, each scheduler unit 1704includes a single dispatch unit 1706 or additional dispatch units 1706.

In at least one embodiment, each SM 1700D, in at least one embodiment,includes, without limitation, register file 1708 that provides a set ofregisters for functional units of SM 1700D. In at least one embodiment,register file 1708 is divided between each of functional units such thateach functional unit is allocated a dedicated portion of register file1708. In at least one embodiment, register file 1708 is divided betweendifferent warps being executed by SM 1700D and register file 1708provides temporary storage for operands connected to data paths offunctional units. In at least one embodiment, each SM 1700D comprises,without limitation, a plurality of L processing cores 1710. In at leastone embodiment, SM 1700D includes, without limitation, a large number(e.g., 128 or more) of distinct processing cores 1710. In at least oneembodiment, each processing core 1710, in at least one embodiment,includes, without limitation, a fully-pipelined, single-precision,double-precision, and/or mixed precision processing unit that includes,without limitation, a floating point arithmetic logic unit and aninteger arithmetic logic unit. In at least one embodiment, floatingpoint arithmetic logic units implement IEEE 754-2008 standard forfloating point arithmetic. In at least one embodiment, processing cores1710 include, without limitation, 64 single-precision (32-bit) floatingpoint cores, 64 integer cores, 32 double-precision (64-bit) floatingpoint cores, and 8 tensor cores.

Tensor cores are configured to perform matrix operations in accordancewith at least one embodiment. In at least one embodiment, one or moretensor cores are included in processing cores 1710. In at least oneembodiment, tensor cores are configured to perform deep learning matrixarithmetic, such as convolution operations for neural network trainingand inferencing. In at least one embodiment, each tensor core operateson a 4×4 matrix and performs a matrix multiply and accumulate operationD=A×B+C, where A, B, C, and D are 4×4 matrices.

In at least one embodiment, matrix multiply inputs A and B are 16-bitfloating point matrices and accumulation matrices C and D are 16-bitfloating point or 32-bit floating point matrices. In at least oneembodiment, tensor cores operate on 16-bit floating point input datawith 32-bit floating point accumulation. In at least one embodiment,16-bit floating point multiply uses 64 operations and results in a fullprecision product that is then accumulated using 32-bit floating pointaddition with other intermediate products for a 4×4×4 matrix multiply.Tensor cores are used to perform much larger two-dimensional or higherdimensional matrix operations, built up from these smaller elements, inat least one embodiment. In at least one embodiment, an API, such asCUDA 9 C++ API, exposes specialized matrix load, matrix multiply andaccumulate, and matrix store operations to efficiently use tensor coresfrom a CUDA-C++ program. In at least one embodiment, at CUDA level,warp-level interface assumes 16×16 size matrices spanning all 32 threadsof warp.

In at least one embodiment, each SM 1700D comprises, without limitation,M SFUs 1712 that perform special functions (e.g., attribute evaluation,reciprocal square root, etc.). In at least one embodiment, SFUs 1712include, without limitation, a tree traversal unit configured totraverse a hierarchical tree data structure. In at least one embodiment,SFUs 1712 include, without limitation, a texture unit configured toperform texture map filtering operations. In at least one embodiment,texture units are configured to load texture maps (e.g., a 2D array oftexels) from memory and sample texture maps to produce sampled texturevalues for use in shader programs executed by SM 1700D. In at least oneembodiment, texture maps are stored in shared memory/L1 cache 1718. Inat least one embodiment, texture units implement texture operations suchas filtering operations using mip-maps (e.g., texture maps of varyinglevels of detail), in accordance with at least one embodiment. In atleast one embodiment, each SM 1700D includes, without limitation, twotexture units.

Each SM 1700D comprises, without limitation, N LSUs 1714 that implementload and store operations between shared memory/L1 cache 1718 andregister file 1708, in at least one embodiment. Each SM 1700D includes,without limitation, interconnect network 1716 that connects each offunctional units to register file 1708 and LSU 1714 to register file1708 and shared memory/L1 cache 1718 in at least one embodiment. In atleast one embodiment, interconnect network 1716 is a crossbar that canbe configured to connect any of functional units to any of registers inregister file 1708 and connect LSUs 1714 to register file 1708 andmemory locations in shared memory/L1 cache 1718.

In at least one embodiment, shared memory/L1 cache 1718 is an array ofon-chip memory that allows for data storage and communication between SM1700D and primitive engine and between threads in SM 1700D, in at leastone embodiment. In at least one embodiment, shared memory/L1 cache 1718comprises, without limitation, 128 KB of storage capacity and is in pathfrom SM 1700D to partition unit. In at least one embodiment, sharedmemory/L1 cache 1718, in at least one embodiment, is used to cache readsand writes. In at least one embodiment, one or more of shared memory/L1cache 1718, L2 cache, and memory are backing stores.

Combining data cache and shared memory functionality into a singlememory block provides improved performance for both types of memoryaccesses, in at least one embodiment. In at least one embodiment,capacity is used or is usable as a cache by programs that do not useshared memory, such as if shared memory is configured to use half ofcapacity, texture and load/store operations can use remaining capacity.Integration within shared memory/L1 cache 1718 enables shared memory/L1cache 1718 to function as a high-throughput conduit for streaming datawhile simultaneously providing high-bandwidth and low-latency access tofrequently reused data, in accordance with at least one embodiment. Inat least one embodiment, when configured for general purpose parallelcomputation, a simpler configuration can be used compared with graphicsprocessing. In at least one embodiment, fixed function graphicsprocessing units are bypassed, creating a much simpler programmingmodel. In general purpose parallel computation configuration, workdistribution unit assigns and distributes blocks of threads directly toDPCs, in at least one embodiment. In at least one embodiment, threads ina block execute same program, using a unique thread ID in calculation toensure each thread generates unique results, using SM 1700D to executeprogram and perform calculations, shared memory/L1 cache 1718 tocommunicate between threads, and LSU 1714 to read and write globalmemory through shared memory/L1 cache 1718 and memory partition unit. Inat least one embodiment, when configured for general purpose parallelcomputation, SM 1700D writes commands that scheduler unit 1704 can useto launch new work on DPCs.

In at least one embodiment, PPU is included in or coupled to a desktopcomputer, a laptop computer, a tablet computer, servers, supercomputers,a smart-phone (e.g., a wireless, hand-held device), personal digitalassistant (“PDA”), a digital camera, a vehicle, a head mounted display,a hand-held electronic device, and more. In at least one embodiment, PPUis embodied on a single semiconductor substrate. In at least oneembodiment, PPU is included in a system-on-a-chip (“SoC”) along with oneor more other devices such as additional PPUs, memory, a reducedinstruction set computer (“RISC”) CPU, a memory management unit (“MMU”),a digital-to-analog converter (“DAC”), and like.

In at least one embodiment, PPU may be included on a graphics card thatincludes one or more memory devices. A graphics card may be configuredto interface with a PCIe slot on a motherboard of a desktop computer. Inat least one embodiment, PPU may be an integrated graphics processingunit (“iGPU”) included in chipset of motherboard.

Inference and/or training logic 615 are used to perform inferencingand/or training operations associated with one or more embodiments.Details regarding inference and/or training logic 615 are provided belowin conjunction with FIGS. 6B and/or 6C. In at least one embodiment, deeplearning application processor is used to train a machine learningmodel, such as a neural network, to predict or infer informationprovided to SM 1700D. In at least one embodiment, SM 1700D is used toinfer or predict information based on a trained machine learning model(e.g., neural network) that has been trained by another processor orsystem or by SM 1700D. In at least one embodiment, SM 1700D may be usedto perform one or more neural network use cases described herein.

In at least one embodiment, a single semiconductor platform may refer toa sole unitary semiconductor-based integrated circuit or chip. In atleast one embodiment, multi-chip modules may be used with increasedconnectivity which simulate on-chip operation, and make substantialimprovements over utilizing a central processing unit (“CPU”) and busimplementation. In at least one embodiment, various modules may also besituated separately or in various combinations of semiconductorplatforms per desires of user.

In at least one embodiment, computer programs in form ofmachine-readable executable code or computer control logic algorithmsare stored in main memory 4ee04 and/or secondary storage. Computerprograms, if executed by one or more processors, enable system 4ee00 toperform various functions in accordance with at least one embodiment. Inat least one embodiment, memory 4ee04, storage, and/or any other storageare possible examples of computer-readable media. In at least oneembodiment, secondary storage may refer to any suitable storage deviceor system such as a hard disk drive and/or a removable storage drive,representing a floppy disk drive, a magnetic tape drive, a compact diskdrive, digital versatile disk (“DVD”) drive, recording device, universalserial bus (“USB”) flash memory, etc. In at least one embodiment,architecture and/or functionality of various previous figures areimplemented in context of CPU 4ee02; parallel processing system 4ee12;an integrated circuit capable of at least a portion of capabilities ofboth CPU 4ee02; parallel processing system 4ee12; a chipset (e.g., agroup of integrated circuits designed to work and sold as a unit forperforming related functions, etc.); and any suitable combination ofintegrated circuit(s).

In at least one embodiment, architecture and/or functionality of variousprevious figures are implemented in context of a general computersystem, a circuit board system, a game console system dedicated forentertainment purposes, an application-specific system, and more. In atleast one embodiment, computer system 4ee00 may take form of a desktopcomputer, a laptop computer, a tablet computer, servers, supercomputers,a smart-phone (e.g., a wireless, hand-held device), personal digitalassistant (“PDA”), a digital camera, a vehicle, a head mounted display,a hand-held electronic device, a mobile phone device, a television,workstation, game consoles, embedded system, and/or any other type oflogic.

In at least one embodiment, parallel processing system 4ee12 includes,without limitation, a plurality of parallel processing units (“PPUs”)4ee14 and associated memories 4ee16. In at least one embodiment, PPUs4ee14 are connected to a host processor or other peripheral devices viaan interconnect 4ee18 and a switch 4ee20 or multiplexer. In at least oneembodiment, parallel processing system 4ee12 distributes computationaltasks across PPUs 4ee14 which can be parallelizable—for example, as partof distribution of computational tasks across multiple graphicsprocessing unit (“GPU”) thread blocks. In at least one embodiment,memory is shared and accessible (e.g., for read and/or write access)across some or all of PPUs 4ee14, although such shared memory may incurperformance penalties relative to use of local memory and registersresident to a PPU 4ee14. In at least one embodiment, operation of PPUs4ee14 is synchronized through use of a command such as _syncthreads( ),wherein all threads in a block (e.g., executed across multiple PPUs4ee14) to reach a certain point of execution of code before proceeding.

Other variations are within spirit of present disclosure. Thus, whiledisclosed techniques are susceptible to various modifications andalternative constructions, certain illustrated embodiments thereof areshown in drawings and have been described above in detail. It should beunderstood, however, that there is no intention to limit disclosure tospecific form or forms disclosed, but on contrary, intention is to coverall modifications, alternative constructions, and equivalents fallingwithin spirit and scope of disclosure, as defined in appended claims.

Use of terms “a” and “an” and “the” and similar referents in context ofdescribing disclosed embodiments (especially in context of followingclaims) are to be construed to cover both singular and plural, unlessotherwise indicated herein or clearly contradicted by context, and notas a definition of a term. Terms “including,” “having,” “including,” and“containing” are to be construed as open-ended terms (meaning“including, but not limited to,”) unless otherwise noted. Term“connected,” when unmodified and referring to physical connections, isto be construed as partly or wholly contained within, attached to, orjoined together, even if there is something intervening. Recitation ofranges of values herein are merely intended to serve as a shorthandmethod of referring individually to each separate value falling withinrange, unless otherwise indicated herein and each separate value isincorporated into specification as if it were individually recitedherein. Use of a set (e.g., a set of items) or subset, unless otherwisenoted or contradicted by context, is to be construed as a nonemptycollection including one or more members. Further, unless otherwisenoted or contradicted by context, a subset of a corresponding set doesnot necessarily denote a proper subset of corresponding set, but subsetand corresponding set may be equal.

Conjunctive language, such as phrases of form “at least one of A, B, andC,” or “at least one of A, B and C,” unless specifically statedotherwise or otherwise clearly contradicted by context, is otherwiseunderstood with context as used in general to present that an item,term, etc., may be either A or B or C, or any nonempty subset of set ofA and B and C. For instance, in illustrative example of a set havingthree members, conjunctive phrases “at least one of A, B, and C” and “atleast one of A, B and C” refer to any of following sets: {A}, {B}, {C},{A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such conjunctive language maynot be intended to imply that certain embodiments require at least oneof A, at least one of B, and at least one of C each to be present. Inaddition, unless otherwise noted or contradicted by context, a pluralityindicates a state of being plural (e.g., a plurality of items indicatesmultiple items). A plurality is at least two items, but can be more whenso indicated either explicitly or by context. Further, unless statedotherwise or otherwise clear from context, based on means based at leastin part on and not based solely on.

Operations of processes described herein can be performed in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context. In at least one embodiment, a process such asthose processes described herein (or variations and/or combinationsthereof) is performed under control of one or more computer systemsconfigured with executable instructions and is implemented as code(e.g., executable instructions, one or more computer programs or one ormore applications) executing collectively on one or more processors, byhardware or combinations thereof. In at least one embodiment, code isstored on a computer-readable storage medium, for example, in form of acomputer program including a plurality of instructions executable by oneor more processors. In at least one embodiment, a computer-readablestorage medium is a non-transitory computer-readable storage medium thatexcludes transitory signals (e.g., a propagating transient electric orelectromagnetic transmission) but includes non-transitory data storagecircuitry (e.g., buffers, cache, and queues) within transceivers oftransitory signals. In at least one embodiment, code (e.g., executablecode or source code) is stored on a set of one or more non-transitorycomputer-readable storage media having stored thereon executableinstructions (or other memory to store executable instructions) that,when executed (in at least one embodiment, as a result of beingexecuted) by one or more processors of a computer system, cause computersystem to perform operations described herein. A set of non-transitorycomputer-readable storage media, in at least one embodiment, includesmultiple non-transitory computer-readable storage media and one or moreof individual non-transitory storage media of multiple non-transitorycomputer-readable storage media lack all of code while multiplenon-transitory computer-readable storage media collectively store all ofcode. In at least one embodiment, executable instructions are executedsuch that different instructions are executed by differentprocessors—for example, a non-transitory computer-readable storagemedium store instructions and a main central processing unit (“CPU”)executes some of instructions while a graphics processing unit (“GPU”)executes other instructions. In at least one embodiment, differentcomponents of a computer system have separate processors and differentprocessors execute different subsets of instructions.

Accordingly, in at least one embodiment, computer systems are configuredto implement one or more services that singly or collectively performoperations of processes described herein and such computer systems areconfigured with applicable hardware and/or software that enableperformance of operations. Further, a computer system that implements atleast one embodiment of present disclosure is a single device and, inanother embodiment, is a distributed computer system including multipledevices that operate differently such that distributed computer systemperforms operations described herein and such that a single device doesnot perform all operations.

Use of any and all examples, or exemplary language (e.g., “such as”)provided herein, is intended merely to better illuminate embodiments ofdisclosure and does not pose a limitation on scope of disclosure unlessotherwise claimed. No language in specification should be construed asindicating any non-claimed element as essential to practice ofdisclosure.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

In description and claims, terms “coupled” and “connected,” along withtheir derivatives, may be used. It should be understood that these termsmay be not intended as synonyms for each other. Rather, in particularexamples, “connected” or “coupled” may be used to indicate that two ormore elements are in direct or indirect physical or electrical contactwith each other. “Coupled” may also mean that two or more elements arenot in direct contact with each other, but yet still co-operate orinteract with each other.

Unless specifically stated otherwise, it may be appreciated thatthroughout specification, references to processing, computing,calculating, determining, or the like, refer to action and/or processesof a computer or computing system, or similar electronic computingdevice, that manipulate and/or transform data represented as physical,such as electronic, quantities within computing system's registersand/or memories into other data similarly represented as physicalquantities within computing system's memories, registers or other suchinformation storage, transmission or display devices.

In a similar manner, a processor may refer to any device or portion of adevice that processes electronic data from registers and/or memory andtransform that electronic data into other electronic data that may bestored in registers and/or memory. As non-limiting examples, “processor”may be a CPU or a GPU. A “computing platform” may include one or moreprocessors. As used herein, “software” processes may include, forexample, software and/or hardware entities that perform work over time,such as tasks, threads, and intelligent agents. Also, each process mayrefer to multiple processes, for carrying out instructions in sequenceor in parallel, continuously or intermittently. Terms “system” and“method” are used herein interchangeably insofar as system may embodyone or more methods and methods may be considered a system.

In present document, references may be made to obtaining, acquiring,receiving, or inputting analog or digital data into a subsystem,computer system, or computer-implemented machine. Obtaining, acquiring,receiving, or inputting analog and digital data can be accomplished in avariety of ways such as by receiving data as a parameter of a functioncall or a call to an application programming interface. In someimplementations, process of obtaining, acquiring, receiving, orinputting analog or digital data can be accomplished by transferringdata via a serial or parallel interface. In another implementation,process of obtaining, acquiring, receiving, or inputting analog ordigital data can be accomplished by transferring data via a computernetwork from providing entity to acquiring entity. References may alsobe made to providing, outputting, transmitting, sending, or presentinganalog or digital data. In various examples, process of providing,outputting, transmitting, sending, or presenting analog or digital datacan be accomplished by transferring data as an input or output parameterof a function call, a parameter of an application programming interfaceor interprocess communication mechanism.

Although discussion above sets forth example implementations ofdescribed techniques, other architectures may be used to implementdescribed functionality, and are intended to be within scope of thisdisclosure. Furthermore, although specific distributions ofresponsibilities are defined above for purposes of discussion, variousfunctions and responsibilities might be distributed and divided indifferent ways, depending on circumstances.

Furthermore, although subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that subject matter claimed in appended claims is notnecessarily limited to specific features or acts described. Rather,specific features and acts are disclosed as exemplary forms ofimplementing the claims.

What is claimed is:
 1. A cooling system for a datacenter, comprising: amultiple mode cooling subsystem comprising two or more of an air-basedcooling subsystem, a liquid-based cooling subsystem, a dielectric-basedcooling subsystem, or a phase-based cooling subsystem, the multiple modecooling subsystem rated for different cooling capacities and adjustableto different cooling requirements of the datacenter within a minimum anda maximum of the different cooling capacities, the multiple mode coolingsubsystem further having a form factor for one or more racks of thedatacenter.
 2. The cooling system of claim 1, further comprising: arefrigerant-based heat transfer subsystem to enable one or more of thephase-based cooling subsystem and the dielectric-based coolingsubsystem; and a coolant-based heat transfer subsystem to enable one ormore of the liquid-based cooling subsystem and the air-based coolingsubsystem.
 3. The cooling system of claim 1, further comprising: adielectric refrigerant to enable the phase-based cooling subsystem andthe dielectric-based cooling subsystem.
 4. The cooling system of claim1, further comprising: a coolant distributor to provide coolant for theair-based cooling subsystem and the liquid-based cooling subsystem. 5.The cooling system of claim 1, further comprising: a first individualcooling subsystem of the multiple mode cooling subsystem havingassociated first looping components to guide a first cooling media to atleast one first rack of the datacenter according to a first heat featureor a first cooling requirement of the at least one first rack; and asecond individual cooling subsystem of the multiple mode coolingsubsystem having associated second looping components to guide a secondcooling media to at least one second rack of the datacenter according toa second heat feature or a second cooling requirement of the at leastone second rack.
 6. The cooling system of claim 5, wherein the firstheat feature or a first cooling requirement is within a first thresholdof the minimum of the different cooling capacities and second heatfeature or a second cooling requirement is within a second threshold ofthe maximum of the different cooling capacities.
 7. The cooling systemof claim 1, further comprising: inlet and outlet lines to a rackmanifold of the one or more racks to enable individual serversassociated with the rack manifold to receive different cooling mediafrom individual cooling subsystems of the multiple mode coolingsubsystem.
 8. The cooling system of claim 1, further comprising: one ormore servers having two or more of an air distribution unit, a coolantdistribution unit, and a refrigerant distribution unit to enabledifferent cooling media to address the different cooling requirements ofthe one or more servers.
 9. The cooling system of claim 1, furthercomprising: a learning subsystem comprising at least one processor toevaluate temperatures within servers of the one or more racks with flowrates of different cooling media, and to provide outputs associated withat least two temperatures to facilitate movement of two cooling media ofthe different cooling media by controlling flow controllers associatedwith the two cooling media to address the different coolingrequirements.
 10. The cooling system of claim 9, further comprising:cold plates associated with components in the servers; the flowcontrollers facilitating the movement of the different cooling mediathrough the cold plates or through the servers; and the learningsubsystem executing a machine learning model to: process thetemperatures using multiple neuron levels of the machine learning modelhaving the temperatures and having prior associated flow rates for thedifferent cooling media; and provide the outputs associated with flowrates for the different cooling media to the flow controllers, theoutputs provided after an evaluation of the prior associated flow ratesof individual cooling media of the different cooling media and priorassociated temperatures.
 11. The cooling system of claim 1, furthercomprising: flow controllers to simultaneously modify flow rates ofdifferent cooling media associated with the multiple mode coolingsubsystem.
 12. The cooling system of claim 1, further comprising: one ormore first flow controllers maintaining a first cooling media ofdifferent cooling media associated with the multiple mode coolingsubsystem at a first flow rate based in part on a first coolingrequirement of the different cooling requirements; and one or moresecond flow controllers modifying a second cooling media of thedifferent cooling media to flow at one or more flow rates based in parton a second cooling requirement of the different cooling requirementsthat is sensed from within one or more servers of the datacenter.
 13. Atleast one processor for a cooling system, comprising: at least one logicunit to control flow controllers associated with a multiple mode coolingsubsystem comprising two or more of an air-based cooling subsystem, aliquid-based cooling subsystem, a dielectric-based cooling subsystem, ora phase-based cooling subsystem, the multiple mode cooling subsystemrated for different cooling capacities and adjustable to differentcooling requirements of the datacenter within a minimum and a maximum ofthe different cooling capacities by a control of cooling mediaassociated with the flow controllers, the cooling system further havinga form factor for one or more racks of a datacenter.
 14. The least oneprocessor of claim 13, further comprising: a learning subsystem toevaluate temperatures within servers of the one or more racks with flowrates of different cooling media, and to provide outputs associated withat least two temperatures to facilitate movement of two cooling media ofthe different cooling media by controlling the flow controllers toaddress the different cooling requirements.
 15. The least one processorof claim 14, further comprising: the learning subsystem executing amachine learning model to: process the temperatures using multipleneuron levels of the machine learning model having the temperatures andhaving prior associated flow rates for the different cooling media; andprovide the outputs associated with flow rates for the two cooling mediato the flow controllers, the outputs provided after an evaluation of theprior associated flow rates of individual cooling media of the differentcooling media and prior associated temperatures.
 16. The at least oneprocessor of claim 14, further comprising: at least one instructionoutput to communicate the outputs associated with the flow controllersto modify a first flow rate of a first individual cooling media, whilemaintaining a second flow rate of a second individual cooling media. 17.The at least one processor of claim 14, further comprising: the at leastone logic unit adapted to receive temperature values from temperaturesensors associated with servers or the one or more racks, and adapted tofacilitate movement of a first individual cooling media and a secondindividual cooling media of the at least two cooling media.
 18. At leastone processor for a cooling system, comprising: at least one logic unitto train one or more neural network having hidden layers of neurons toevaluate temperatures and flow rates that are associated with at leasttwo cooling media of a multiple mode cooling subsystem comprising two ormore of an air-based cooling subsystem, a liquid-based coolingsubsystem, a dielectric-based cooling subsystem, or a phase-basedcooling subsystem, the multiple mode cooling subsystem rated fordifferent cooling capacities and adjustable to different coolingrequirements of the datacenter within a minimum and a maximum of thedifferent cooling capacities, the cooling system further having a formfactor for one or more racks of a datacenter.
 19. The at least oneprocessor of claim 18, further comprising: the at least one logic unitto output an instruction associated with at least two temperatures tofacilitate movement of the at least two cooling media by controllingflow controllers associated with the at least two cooling media toaddress the different cooling requirements.
 20. The at least oneprocessor of claim 19, further comprising: at least one instructionoutput to communicate the outputs with the flow controllers to modify afirst flow rate of a first cooling media of the at least two coolingmedia, while maintaining a second flow rate of a second individualcooling media.
 21. The at least one processor of claim 19, furthercomprising: the at least one logic unit adapted to receive temperaturevalues from temperature sensors associated with servers of the one ormore racks, and adapted to facilitate movement of a first individualcooling media and a second individual cooling media of the at least twocooling media.
 22. A datacenter cooling system, comprising: at leastprocessor to train one or more neural network having hidden layers ofneurons to evaluate temperatures and flow rates that are associated withat least two cooling media of a multiple mode cooling subsystemcomprising two or more of an air-based cooling subsystem, a liquid-basedcooling subsystem, a dielectric-based cooling subsystem, or aphase-based cooling subsystem, the multiple mode cooling subsystem ratedfor different cooling capacities and adjustable to different coolingrequirements of the datacenter within a minimum and a maximum of thedifferent cooling capacities, the cooling system further having a formfactor for one or more racks of a datacenter.
 23. The datacenter coolingsystem of claim 22, further comprising: the at least one processor tooutput an instruction associated with at least two temperatures tofacilitate movement of the at least two cooling media by controllingflow controllers associated with the at least two cooling media toaddress the different cooling requirements.
 24. The datacenter coolingsystem of claim 23, further comprising: at least an instruction outputof the at least one processor to communicate the output with the flowcontrollers to modify a first flow rate of a first cooling media of theat least two cooling media, while maintaining a second flow rate of asecond individual cooling media.
 25. The datacenter cooling system ofclaim 22, further comprising: the at least one processor adapted toreceive temperature values from temperature sensors associated withservers of the one or more racks, and adapted to facilitate movement ofa first individual cooling media and a second individual cooling mediaof the at least two cooling media.
 26. A method for cooling adatacenter, comprising: providing a multiple mode cooling subsystemcomprising two or more of an air-based cooling subsystem, a liquid-basedcooling subsystem, a dielectric-based cooling subsystem, or aphase-based cooling subsystem, the multiple mode cooling subsystem ratedfor different cooling capacities and adjustable to different coolingrequirements of the datacenter within a minimum and a maximum of thedifferent cooling capacities, the cooling system further having a formfactor for one or more racks of the datacenter.
 27. The method of claim26, further comprising: enabling flow controllers to receive input fromtemperature sensors associated with one or more components withinservers of the one or more racks of the datacenter; and enabling theflow controllers to facilitate movement of two cooling media bycontrolling the flow controllers associated with the two cooling mediato address the different cooling requirements.
 28. The method of claim26, further comprising: enabling one or more of the phase-based coolingsubsystem and the dielectric-based cooling subsystem using arefrigerant-based heat transfer subsystem; and enabling one or more ofthe air-based cooling subsystem and the liquid-based cooling subsystemusing a coolant-based heat transfer subsystem.
 29. The method of claim26, further comprising: enabling the phase-based cooling subsystem andthe dielectric cooling subsystem using a dielectric refrigerant.
 30. Themethod of claim 26, further comprising: guiding a first cooling media toat least one first rack of the datacenter according to a first heatfeature or a first cooling requirement of the at least one first rackusing a first individual cooling subsystem of the multiple mode coolingsubsystem having associated first looping components; and guiding asecond cooling media to at least one second rack of the datacenteraccording to a second heat feature or a second cooling requirement ofthe at least one second rack using a second individual cooling subsystemof the multiple mode cooling subsystem having associated second loopingcomponents.